Some thoughts on covidHubUtils development

[nikosbosse]

file paths

• file paths should maybe use the here package. Sam has strong opinions on that - apparently it makes things much easier when running something on a server. e.g. in the get truth functions

documentation

• documentation could maybe be more verbose to help outsiders understand it
• examples in the documenation would help a lot
• some of the documentation is US specific

Function arguments

• Maybe it would be clearer to set arguments that can be left blank to NULL everywhere, so that users know they don't need to provide it? This is the case in some functions, but not in all. Ideally this should be handled consistently across functions

Zoltar connections

• I think it would be good to avoid forced zoltar connections where possible. Options:
  • We could e.g. save the valid targets etc. locally? (except for the timezeros).
  • Or alternatively, make the zoltar checks optional?

load truth

• add functionality to get different target_variables at once
• maybe add functionality for adjustments, i.e. don't give negative cases?

load forecasts

• The load_forecasts functions are a bit confusing in their structure and it isn't immediately clear which calls which.
  • Maybe the naming could be adapted to make that a bit clearer?
  • Maybe we could just have one load_forecasts function instead of another latest function (given that the latest function also needs an explicit forecast date)? Searching within a window also makes sense for the local hub repo in general

additional features that would be nice

• code for creating model ensembles

Thoughts I had earlier that may be outdated and other functions I haven't looked at in detail yet

• plot_forecast.R

  • locations are currently hard-coded

• get_plot_forecast_data.R

  • fips codes hard coded

• get_all_models.R

  • remote_repo option has US hard-coded
  • zoltar option has US hard-coded
  • some more documentation for what it actually does might be helpful. E.g.: where does the path to the local repo have to point to?

• get_hub_locations.R

  • has US locations hard-coded
  • current European file is here: https://github.com/epiforecasts/covid19-forecast-hub-europe/blob/main/template/locations\_eu.csv and doesn't have abbreviations - so maybe the formatting would also have to be changed

• get_model_designations.R

  • documentations is specific to US hub
  • not sure, but I think it can't handle spaces in your file names?

• get_truth.R

  • unsure whether this is only used by maintainers?
  • preprocess_truth_for_zoltar has only deaths and that is hard-coded. If we want to use it it probably needs some adaptions.

• date_management.R

  • maybe some documentation

[elray1]

r.e. file paths in the functions get_truth.R -- I agree that using the here package is useful, but I think that use of here belongs in the script calling those functions rather than in the functions themselves

[nickreich]

Re: documentation- we added issues #192 #193 #194 #195 to improve documentation.

[elray1]

Nikos, do you have additional specific suggestions for places where documentation could be more verbose?

[elray1]

r.e.: "maybe add functionality for adjustments, i.e. don't give negative cases?"

For now, we've decided adjusting truth data is out of scope. There is some functionality related to this in the covidData package, and we're working towards adding support for European hub truth data there.

[nickreich]

Issue #198 created to have a designated place to iterate on re-design of load_forecasts.

[nickreich]

We would happily accept a PR to create simple ensembles. given our existing functionality in covidEnsembles, we are not going to prioritize this development.