Accept parameters through a YAML file using sacred

[paola-g]

https://sacred.readthedocs.io/en/stable/configuration.html

(but check that it will still work as a command line tool for backward compatibility)

An example from CoPilot

The Sacred framework is a Python tool designed to help you configure, organize, log, and reproduce experiments. It is particularly useful in the context of machine learning and scientific computing. Sacred can be easily integrated with YAML configuration files to manage your experiment's parameters.

1. Install the Sacred and PyYAML libraries if you haven't already:

   pip install sacred pyyaml

2. Create a Python script that uses Sacred to read the configuration from a YAML file.

Here is an example of how you can achieve this:

import yaml
from sacred import Experiment
from sacred.observers import FileStorageObserver

# Create a Sacred experiment
ex = Experiment('example_experiment')

# Add an observer to log the results
ex.observers.append(FileStorageObserver.create('my_runs'))

@ex.config
def my_config():
    # Default configuration parameters
    foo = 1
    bar = "default"
    baz = False

@ex.named_config
def from_yaml():
    with open('config.yaml', 'r') as f:
        cfg = yaml.safe_load(f)
    # Update the configuration with values from the YAML file
    for k, v in cfg.items():
        globals()[k] = v

@ex.automain
def main(foo, bar, baz):
    # Your experiment logic here
    print(f'foo: {foo}')
    print(f'bar: {bar}')
    print(f'baz: {baz}')

3. Create a YAML configuration file named config.yaml with the following content:

foo: 42
bar: hello
baz: true

4. Run the Sacred experiment, specifying the YAML configuration:

python experiment_sacred.py with from_yaml

Sacred will read the parameters from the YAML file and use them in the experiment. This allows you to manage and reproduce your experiments more easily. The FileStorageObserver will log the results of each run, allowing you to analyze and reproduce them later.

Combining Sacred and argparse

The script will first check if a YAML configuration file is provided. If it is, Sacred will be used to read the configuration from the YAML file. Otherwise, argparse will be used to parse the command-line arguments.

Here is how you can structure your script to achieve this:

import argparse
import yaml
from sacred import Experiment
from sacred.observers import FileStorageObserver

# Create a Sacred experiment
ex = Experiment('example_experiment')

# Add an observer to log the results
ex.observers.append(FileStorageObserver.create('my_runs'))

@ex.config
def my_config():
    foo = 1
    bar = "default"
    baz = False

def load_yaml_config(yaml_file):
    with open(yaml_file, 'r') as f:
        return yaml.safe_load(f)

@ex.named_config
def from_yaml():
    config_file = 'config.yaml'
    cfg = load_yaml_config(config_file)
    for k, v in cfg.items():
        globals()[k] = v

@ex.automain
def main(foo, bar, baz):
    print(f'foo: {foo}')
    print(f'bar: {bar}')
    print(f'baz: {baz}')

def main_argparse():
    parser = argparse.ArgumentParser(description="Parse command line arguments.")
    parser.add_argument('--foo', type=int, help="Foo argument")
    parser.add_argument('--bar', type=str, help="Bar argument")
    parser.add_argument('--baz', type=bool, help="Baz argument")
    args = parser.parse_args()
    
    print(f'foo: {args.foo}')
    print(f'bar: {args.bar}')
    print(f'baz: {args.baz}')

if __name__ == "__main__":
    parser = argparse.ArgumentParser(add_help=False)
    parser.add_argument('--config', type=str, help="Path to the YAML configuration file.")
    args, remaining_argv = parser.parse_known_args()

    if args.config:
        ex.run(named_configs=['from_yaml'])
    else:
        main_argparse()

Explanation

1. Sacred Configuration:

   • The @ex.config decorator defines the default configuration.
   • The from_yaml named configuration loads parameters from a YAML file.
   • The main function within the @ex.automain decorator runs the experiment when using Sacred.

2. Argument Parsing with argparse:

   • The main_argparse function uses argparse to parse command-line arguments and print them.

3. Main Conditional Logic:

   • The script first checks for a --config argument to determine if a YAML configuration file is provided.
   • If a YAML file is provided, Sacred kicks in and uses the named configuration from_yaml.
   • If no YAML file is provided, the script falls back to using argparse to parse the command-line arguments.

Example Usage

1. Using the YAML configuration file:

   python experiment_sacred_argparse.py --config config.yaml

2. Using command-line arguments:

   python experiment_sacred_argparse.py --foo 10 --bar world --baz false

This setup ensures that your script can flexibly handle both YAML configurations and command-line arguments.