Authors meeting - 2023-09-08

[RaphaelS1]

Agenda

• outline/hierarchy
• Notation: Use the same as in mlr3book?
• P year prediction as use case for SA
• if a problem is set up wrongly (i.e. treat interval censoring as right censored, etc) -> performance metrics won't tell you
• Intro: COVID example already outdated? In general, maybe not refer to current events/references as these age quickly
• "Survival analysis (‘survival’) is a unique field of study in Statistics as it includes the added difficulty of ‘censoring’." -> and truncation (or incomplete observations more generally)
• Bibliography
• do we want to continue to use the categories defined in DL overview paper to classify capabilities of different methods?

[RaphaelS1]

1. Outline/hierarchy

[RaphaelS1]

• Worried about having evaluation too late -- need it for some models (e.g., boosting, random forests)
• Reduction techniques shouldn't be combined with evaluation -- too important
• We could use signposting to show chapters than can be skipped or ones that relate to each other and must be read
• e.g., When introducing random forests for log-rank splitting add "See Section X.Y method"

Table of Contents:

0. Preface
1. Introduction

Part I: Survival Analysis and Machine Learning

2. MLSA: The Whole Game
3. Statistical Learning
4. Survival Analysis

Part II: Evaluation

(common -> exotic)

5. Some abstract high-level methods chapter
6. Survival Time Measures
7. Continuous Ranking Measures
8. Distribution Measures by Calibration
9. Distribution Measures by Scoring Rules

Part III: Models

10. Classical
11. Tree-Based
12. SVMs
13. Boosting
14. Neural Networks
15. Alternative Methods

Part IV: Reduction Techniques

16. Discrete Time Survival Analysis (classification)
17. Poisson
18. Pseudo-values (regression, imputation, deletion)
19. Competing Risks (pipeline)

(Part V: Extensions and Outlook)

20. Software
21. What's next?

[RaphaelS1]

@adibender as I was implementing this ToC I got to thinking that after many pages on measures it'll be useful to have a chapter called something like "Choosing survival measures" to help people actually put it into practice and also to address common pitfalls (e.g., C-hacking) if we'd rather not have them in the main body of the text 🤷🏻

Then I wondered if we should do the same with model selection and maybe have a chapter on benchmarking? Doesn't need to be fully applied (i.e., with code) but could be code to have something somewhere that rounds up all the content and talks about trade-offs in model selection with respect to different measures etc.

[RaphaelS1]

2. Notation: Use the same as in mlr3book?

[RaphaelS1]

Send suggestions for papers with good notation for ML and SA (see Bernd paper on HPO and ML)

[RaphaelS1]

3. P year prediction as use case for SA

[RaphaelS1]

Mention somewhere that SA can be used to solve T-year prediciton problems (i.e., see if we can get classif users over to SA).
Also SA can be used for censoring/truncation

[RaphaelS1]

4. if a problem is set up wrongly (i.e. treat interval censoring as right censored, etc) -> performance metrics won't tell you

[RaphaelS1]

Make sure intro is clear about censoring/truncation and that metrics can't highlight if this is setup wrong - analogously to hypothesis testing not testing the result but hypothesis, p-hacking, etc.

[RaphaelS1]

Measures for right-censored data used when internal etc. should've been used instead.

[RaphaelS1]

Same as dependent/independent censoring and measures problem

[RaphaelS1]

5. Intro: COVID example already outdated? In general, maybe not refer to current events/references as these age quickly

[RaphaelS1]

6. "Survival analysis (‘survival’) is a unique field of study in Statistics as it includes the added difficulty of ‘censoring’." -> and truncation (or incomplete observations more generally)

[RaphaelS1]

7. Bibliography

[RaphaelS1]

Just copy/paste references in

[RaphaelS1]

8. do we want to continue to use the categories defined in DL overview paper to classify capabilities of different methods?