difference between baseline survival from baseline_survival_ and plot_partial_effects_on_outcome

[takra]

Hello, @CamDavidsonPilon, colleagues
Can you help me please with such a question, In my study I tried to estimate survival_baseline with 2 methods from lifelines:

1. cph = CoxPHFitter()
   cph.baseline_survival_
2. cph.plot_partial_effects_on_outcome
   and estimates were different.
   First one seems to be real survival_baseline, but the second one is coincident with one of the group. Look in attached files for problem illustration.
   lifelines_baseline_surv_calculation.zip
   The graph seems to be coincident with graphs in David G. Kleinbaum, Mitchel Klein auth. Survival Analysis A Self-Learning Text.
   
   So, why the estimates are different?

[CamDavidsonPilon]

Hi @takra
The baseline survival is only the time-varying part of this equation, specifically $\lambda_0(t)$. When we use partial_effects_on_outcome, we are plotting $\lambda_0(t) \cdot \exp(\beta \cdot \text{some baseline values})$. These baseline values come from the data - could be the median salary, the mode gender, etc. In your case, the baseline values coincide with a group (you can view the baseline values with cph._central_values).

Does that make sense?

[takra]

Hi @CamDavidsonPilon, thank you for reply. I'll try to explain my question by steps.

The baseline survival is only the time-varying part of this equation, specifically
$\lambda_0(t)$.

I agree with statement and I used similar equations in code to estimate survival curves for groups based on baseline_survival_ from CoxPHFitter model. As far as I understand
$baseline survival = \lambda_0(t)^{exp(\beta * \overline{group})}$ - adjusted survival curve for all covariates (in this case - for group), $\overline{group}$ - group mean value
So with known baseline_survival_ one can estimate survival curves for different covariate values with equation:
$\lambda(t,group) = \lambda_0(t)^{exp(\beta * group)}$
or
$\lambda(t,group) = baselinesurvival^{exp(\beta * (group-\overline{group}))}$
So baseline_survival_ from CoxPHFitter is adjusted for all covariates survival curve and its sense is understood.
Achieved from equations survival curves are coincident with plot_partial_effects_on_outcome, so I expected baseline values to be the same as from CoxPHFitter, but it differ. So baseline_survival_ from these 2 cases have different sense. Maybe it is silly question but I don't understand why? what for? how I can use it? Or may be it doesn't matter. If one know baseline_values (cph_central_values) (choosed in anyway) one can calculate from this baseline any survival curve.

[takra]

@CamDavidsonPilon, thanks to your's previos answer about cph_central_values I understand how to make coincident SF achieved by equations based on baseline_survival_ CoxPHFitter and SF by partial_effects_on_outcome in more complex stratified models
lifelines_baseline_surv_calculation.zip. But the problem is that it differs from original and I can't understand why.


description

UPD1: I remember that breslow is applied in book as tie break method but efron in lifelines, but it can't explain difference. I've done same calculations in Matlab with Efron and Breslow. And you can notice that graph with Efron is completely the same as in lifelines, graph with Breslow slightly differs from Efron and dramatically differs from the book's graph, So is any mistake in my equations for survival curves?

[CamDavidsonPilon]

Do I understand correctly that "male Placebo" in lifelines looks like "Treatment, male" in the book? It could be a coding error? That is, 0 should be 1 somewhere?

The green line is throwing me off - that's not at all in the book diagram. Do you agree?

[takra]

Do I understand correctly that "male Placebo" in lifelines looks like "Treatment, male" in the book? It could be a coding error? That is, 0 should be 1 somewhere?

No, it doesn't. You can distinguish male and female by times of events, and treatment always higher than placebo, so not so simple mistake. May be there is mistake in equation, but they are coincident with partial_effects_on_outcome.

The green line is throwing me off - that's not at all in the book diagram. Do you agree?

I won't be so categorical. If one variate value under exponent he can achieve similar curve... And what about equations I've used. Are they correct?

@CamDavidsonPilon

[CamDavidsonPilon]

Can you clarify (simplify) what questions you are asking me? Currently I'm confused.

[takra]

Ok, let me try. Suppose we have survival problem with 3 covariates: group, sex and logWBC. Group and sex are binary, logWBC - continuous. For sex PH assumption is violated, so we consider stratified on sex Cox model with no interaction. The purpose is to estimate survival curves for groups adjusted with mean value for logWBC. So questions:

1. Is it correct to estimate purpose curves with lifelines in such a way?
   cph = CoxPHFitter()
cph.fit(data[['time','status','group','sex','logWBC']], duration_col = 'time', event_col = 'status', strata='sex')
cph.summary
cph.plot_partial_effects_on_outcome(['group','logWBC'],[[0,np.round(data['logWBC'].mean(),3)],[1,np.round(data['logWBC'].mean(),3)]],False,'survival_function',linewidth=3)

2. Is it correct to estimate purpose curves with next equations considering known baseline survivals for stratas from cph.baseline_survival_:
   $S_{female-treatment}$ = $baseline.survival.female^{exp(coeff_{group} * (0-\overline{group}))}$
   $S_{female-placebo}$ = $baseline.survival.female^{exp(coeff_{group} * (1-\overline{group}))}$
   $S_{male-treatment}$ = $baseline.survival.male^{exp(coeff_{group} * (0-\overline{group}))}$
   $S_{male-placebo}$ = $baseline.survival.male^{exp(coeff_{group} * (1-\overline{group}))}$

3. If answer on 1 question is positive then may be you have any ideas why it differs so much from book's originals?