How to find confidence interval for AFT model's predictions?

[Reddon6]

I'm currently following the website's example of fitting the rossi dataset with the WeibullAFTFitter. Once we predict, say, the survival function, how do I give a 95% confidence interval on the survival function? I see from print_summary() that there are exp(coef) upper/lower 95% columns for the parameters. How do I use that to determine the confidence interval for the survival function?

Do I substitute the upper/lower parameter values into the AFT model's survival function formula to calculate the bounds?

Edit: I found the author asking this question previously (https://stats.stackexchange.com/questions/135749/confidence-intervals-of-fitted-weibull-survival-function). I would like to know if this approach can be used for an AFT model. Also, the literature cited in the answer seems to suggest that the approach is meant for exponential distributions (Weibull with shape parameter=1).

Edit 2: Best I can find is from the Survivor function section of this site https://cran.r-project.org/web/packages/ciTools/vignettes/ciTools_survreg_vignette.html

[CamDavidsonPilon]

Hi @Reddon6 - it's not obvious to me how do to this - which is why I've held off on implementing anything. I'm assuming you want to create prediction intervals for different subjects?

The technique in the stats.stackexchange post won't extend to an AFT model, too.

The R lib you linked looks like a good reference though, and seems to favour the delta method (which is used internally for lifelines, as well). If you are familiar with the delta method, all the pieces are available to mimic what they are doing: ex: aft.variance_matrix_, and aft._survival_function (and using a tool like autograd). It would probably be a headache to implement though - so not sure I would go this route for yourself.

Alternatively is to bootstrap - this is simple but time consuming if you have many subjects!

[Reddon6]

Thanks for the response! I'm actually interested in the conditional survival probability (conditional_after) of an average subject.

I did not use the autograd tool but instead tried to find the closed-form partial derivatives of the survival function to mimic what they are doing with aft.variance_matrix_ as well. However, my bounds came out to be absurdly wide. Perhaps I'll try autograd next.

Bootstrap sounds interesting. Does it mean that I resample the data many times and predict a survival probability each time, and then find the 95% intervals from there?

[CamDavidsonPilon]

Does it mean that I resample the data many times and predict a survival probability each time, and then find the 95% intervals from there?

Yup that's right!