Skip to content

Conversation

@frazane
Copy link
Contributor

@frazane frazane commented Oct 17, 2025

The preparation of the inference workspace and the execution itself are refactored into separate rules. Closes #65.

Motivation

  • separation of concerns improves the workflow's clarity and traceability
  • we get a single inference execution rule

Nice extra: debugging inference becomes easier. Starting from the top-level directory, we can now run a single case with

cd output/data/runs/<run-id>/
uv sync && cd <YYYYmmHHMM>/
uv run anemoi-inference run config.yaml

Summary of changes:

  • removed rules inference_forecaster and inference_interpolator
  • created rule prepare_inference_forecaster to set up a forecaster's inference workspace, including the updated config and the resources
  • created rule prepare_inference_interpolation to set up an interpolator's inference workspace, including the updated config, the resources and the parent forecaster's directory (note: this is also created when the "forecaster" is actually the analysis, but we create a placeholder - this is because we cannot conditionally add or remove an output from a rule)
  • created rule execute_inference which will be the same for any kind of model executed with anemoi-inference run config.yaml
  • added a logging setup helper in common.smk so it can be used in a rule's run directive
  • updated the base anemoi-inference version requirements to >=0.8.0,<0.9.0

@frazane frazane marked this pull request as ready for review October 18, 2025 14:29
@frazane frazane force-pushed the feat/decouple-inference-preparation-and-execution branch from 1175afc to f1c6b13 Compare October 21, 2025 14:41
@frazane
Copy link
Contributor Author

frazane commented Oct 21, 2025

@dnerini I refactored the run directives into a script as suggested. It's an improvement, but there's a catch: now whenever we change that script, it will rerun all rules depending on it. So if we update the part for the interpolator, also all forecasters will rerun, unless ones uses the --touch snakemake option first. Should we further refactor into separate scripts, or maybe into a shell directive with a python script.py call like elsewhere?

@frazane frazane merged commit e7f12b2 into main Oct 22, 2025
4 checks passed
@dnerini dnerini deleted the feat/decouple-inference-preparation-and-execution branch October 22, 2025 10:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

3 participants