-Descriptively, we noted apparently similar predictive performance between mechanistic, semi-mechanistic, and statistical models. These model structures appeared to perform relatively worse than agent-based and "other" models. For example, in univariate analysis, the partial effect for statistical models forecasting deaths underperformed by `r table_effects["Deaths_Unadjusted_Statistical","value"]`, (95%CI `r table_effects["Deaths_Unadjusted_Statistical", "lower_2.5"]`-`r table_effects["Deaths_Unadjusted_Statistical", "upper_97.5"]`) compared to average, while agent-based models performed better than average (`r table_effects["Deaths_Unadjusted_Agent-based","value_ci"]`). However, variation in performance overlapped between all model structures, and we noted relative differences between models may have varied over time (Figure \@ref(fig:scores-over-time)). For example, over summer 2021 all model types saw worsening performance coinciding with the introduction of the Delta variant across Europe, but this decline was most marked among statistical models of death outcomes compared to any other model type.
0 commit comments