Skip to content

Simulation logging can't distinguish between unique simulation contexts #282

@collijk

Description

@collijk

vivarium is structured to allow multiple simulation contexts to live in a single python process, but logging is done in interactive settings with global loggers. This means log messages can't distinguish between simulations if they're run from the same Python process. E.g.

>>> from vivarium.examples.disease_model import get_disease_model_simulation
>>> sim1 = get_disease_model_simulation()
2023-02-25 14:54:45.034 | DEBUG    | vivarium.framework.values:register_value_modifier:392 - Registering metrics.1.results_manager.get_results as modifier to metrics
2023-02-25 14:54:45.035 | DEBUG    | vivarium.framework.values:_register_value_producer:338 - Registering value pipeline mortality_rate
2023-02-25 14:54:45.035 | DEBUG    | vivarium.framework.values:_register_value_producer:338 - Registering value pipeline lower_respiratory_infections.cause_specific_mortality_rate
2023-02-25 14:54:45.035 | DEBUG    | vivarium.framework.values:register_value_modifier:392 - Registering mortality_rate.1.diseasemodel.lower_respiratory_infections.delete_cause_specific_mortality as modifier to mortality_rate
2023-02-25 14:54:45.035 | DEBUG    | vivarium.framework.values:register_value_modifier:392 - Registering metrics.2.diseasemodel.lower_respiratory_infections.metrics as modifier to metrics
2023-02-25 14:54:45.035 | DEBUG    | vivarium.framework.values:_register_value_producer:338 - Registering value pipeline susceptible_to_lower_respiratory_infections.excess_mortality_rate
2023-02-25 14:54:45.035 | DEBUG    | vivarium.framework.values:_register_value_producer:338 - Registering value pipeline susceptible_to_lower_respiratory_infections.excess_mortality_rate.population_attributable_fraction
2023-02-25 14:54:45.035 | DEBUG    | vivarium.framework.values:register_value_modifier:392 - Registering mortality_rate.2.diseasestate.susceptible_to_lower_respiratory_infections.add_in_excess_mortality as modifier to mortality_rate
2023-02-25 14:54:45.036 | DEBUG    | vivarium.framework.values:_register_value_producer:338 - Registering value pipeline infected_with_lower_respiratory_infections.incidence_rate
2023-02-25 14:54:45.036 | DEBUG    | vivarium.framework.values:_register_value_producer:338 - Registering value pipeline infected_with_lower_respiratory_infections.incidence_rate.population_attributable_fraction
2023-02-25 14:54:45.036 | DEBUG    | vivarium.framework.values:_register_value_producer:338 - Registering value pipeline infected_with_lower_respiratory_infections.excess_mortality_rate
2023-02-25 14:54:45.036 | DEBUG    | vivarium.framework.values:_register_value_producer:338 - Registering value pipeline infected_with_lower_respiratory_infections.excess_mortality_rate.population_attributable_fraction
2023-02-25 14:54:45.036 | DEBUG    | vivarium.framework.values:register_value_modifier:392 - Registering mortality_rate.3.diseasestate.infected_with_lower_respiratory_infections.add_in_excess_mortality as modifier to mortality_rate
2023-02-25 14:54:45.036 | DEBUG    | vivarium.framework.values:_register_value_producer:338 - Registering value pipeline infected_with_lower_respiratory_infections.remission_rate
2023-02-25 14:54:45.037 | DEBUG    | vivarium.framework.values:_register_value_producer:338 - Registering value pipeline infected_with_lower_respiratory_infections.remission_rate.population_attributable_fraction
2023-02-25 14:54:45.037 | DEBUG    | vivarium.framework.values:_register_value_producer:338 - Registering value pipeline child_wasting.base_proportion_exposed
2023-02-25 14:54:45.037 | DEBUG    | vivarium.framework.values:_register_value_producer:338 - Registering value pipeline child_wasting.proportion_exposed
2023-02-25 14:54:45.037 | DEBUG    | vivarium.framework.values:_register_value_producer:338 - Registering value pipeline child_wasting.exposure
2023-02-25 14:54:45.037 | DEBUG    | vivarium.framework.values:_register_value_producer:338 - Registering value pipeline effect_of_child_wasting_on_infected_with_lower_respiratory_infections.incidence_rate.relative_risk
2023-02-25 14:54:45.038 | DEBUG    | vivarium.framework.values:register_value_modifier:392 - Registering infected_with_lower_respiratory_infections.incidence_rate.population_attributable_fraction.1.effect_of_child_wasting_on_infected_with_lower_respiratory_infections.incidence_rate.population_attributable_fraction as modifier to infected_with_lower_respiratory_infections.incidence_rate.population_attributable_fraction
2023-02-25 14:54:45.038 | DEBUG    | vivarium.framework.values:register_value_modifier:392 - Registering infected_with_lower_respiratory_infections.incidence_rate.1.effect_of_child_wasting_on_infected_with_lower_respiratory_infections.incidence_rate.rate_adjustment as modifier to infected_with_lower_respiratory_infections.incidence_rate
2023-02-25 14:54:45.038 | DEBUG    | vivarium.framework.values:register_value_modifier:392 - Registering child_wasting.proportion_exposed.1.sqlns.intervention_effect as modifier to child_wasting.proportion_exposed
2023-02-25 14:54:45.038 | DEBUG    | vivarium.framework.values:_register_value_producer:338 - Registering value pipeline sqlns.effect_size
2023-02-25 14:54:45.038 | DEBUG    | vivarium.framework.values:register_value_modifier:392 - Registering metrics.3.observer.metrics as modifier to metrics
2023-02-25 14:54:45.038 | DEBUG    | vivarium.framework.values:_register_value_producer:338 - Registering value pipeline metrics
>>> sim2 = get_disease_model_simulation()
2023-02-25 14:54:56.029 | DEBUG    | vivarium.framework.values:register_value_modifier:392 - Registering metrics.1.results_manager.get_results as modifier to metrics
2023-02-25 14:54:56.029 | DEBUG    | vivarium.framework.values:_register_value_producer:338 - Registering value pipeline mortality_rate
2023-02-25 14:54:56.029 | DEBUG    | vivarium.framework.values:_register_value_producer:338 - Registering value pipeline lower_respiratory_infections.cause_specific_mortality_rate
2023-02-25 14:54:56.030 | DEBUG    | vivarium.framework.values:register_value_modifier:392 - Registering mortality_rate.1.diseasemodel.lower_respiratory_infections.delete_cause_specific_mortality as modifier to mortality_rate
2023-02-25 14:54:56.030 | DEBUG    | vivarium.framework.values:register_value_modifier:392 - Registering metrics.2.diseasemodel.lower_respiratory_infections.metrics as modifier to metrics
2023-02-25 14:54:56.030 | DEBUG    | vivarium.framework.values:_register_value_producer:338 - Registering value pipeline susceptible_to_lower_respiratory_infections.excess_mortality_rate
2023-02-25 14:54:56.030 | DEBUG    | vivarium.framework.values:_register_value_producer:338 - Registering value pipeline susceptible_to_lower_respiratory_infections.excess_mortality_rate.population_attributable_fraction
2023-02-25 14:54:56.030 | DEBUG    | vivarium.framework.values:register_value_modifier:392 - Registering mortality_rate.2.diseasestate.susceptible_to_lower_respiratory_infections.add_in_excess_mortality as modifier to mortality_rate
2023-02-25 14:54:56.030 | DEBUG    | vivarium.framework.values:_register_value_producer:338 - Registering value pipeline infected_with_lower_respiratory_infections.incidence_rate
2023-02-25 14:54:56.030 | DEBUG    | vivarium.framework.values:_register_value_producer:338 - Registering value pipeline infected_with_lower_respiratory_infections.incidence_rate.population_attributable_fraction
2023-02-25 14:54:56.031 | DEBUG    | vivarium.framework.values:_register_value_producer:338 - Registering value pipeline infected_with_lower_respiratory_infections.excess_mortality_rate
2023-02-25 14:54:56.031 | DEBUG    | vivarium.framework.values:_register_value_producer:338 - Registering value pipeline infected_with_lower_respiratory_infections.excess_mortality_rate.population_attributable_fraction
2023-02-25 14:54:56.031 | DEBUG    | vivarium.framework.values:register_value_modifier:392 - Registering mortality_rate.3.diseasestate.infected_with_lower_respiratory_infections.add_in_excess_mortality as modifier to mortality_rate
2023-02-25 14:54:56.031 | DEBUG    | vivarium.framework.values:_register_value_producer:338 - Registering value pipeline infected_with_lower_respiratory_infections.remission_rate
2023-02-25 14:54:56.031 | DEBUG    | vivarium.framework.values:_register_value_producer:338 - Registering value pipeline infected_with_lower_respiratory_infections.remission_rate.population_attributable_fraction
2023-02-25 14:54:56.031 | DEBUG    | vivarium.framework.values:_register_value_producer:338 - Registering value pipeline child_wasting.base_proportion_exposed
2023-02-25 14:54:56.031 | DEBUG    | vivarium.framework.values:_register_value_producer:338 - Registering value pipeline child_wasting.proportion_exposed
2023-02-25 14:54:56.031 | DEBUG    | vivarium.framework.values:_register_value_producer:338 - Registering value pipeline child_wasting.exposure
2023-02-25 14:54:56.032 | DEBUG    | vivarium.framework.values:_register_value_producer:338 - Registering value pipeline effect_of_child_wasting_on_infected_with_lower_respiratory_infections.incidence_rate.relative_risk
2023-02-25 14:54:56.032 | DEBUG    | vivarium.framework.values:register_value_modifier:392 - Registering infected_with_lower_respiratory_infections.incidence_rate.population_attributable_fraction.1.effect_of_child_wasting_on_infected_with_lower_respiratory_infections.incidence_rate.population_attributable_fraction as modifier to infected_with_lower_respiratory_infections.incidence_rate.population_attributable_fraction
2023-02-25 14:54:56.032 | DEBUG    | vivarium.framework.values:register_value_modifier:392 - Registering infected_with_lower_respiratory_infections.incidence_rate.1.effect_of_child_wasting_on_infected_with_lower_respiratory_infections.incidence_rate.rate_adjustment as modifier to infected_with_lower_respiratory_infections.incidence_rate
2023-02-25 14:54:56.032 | DEBUG    | vivarium.framework.values:register_value_modifier:392 - Registering child_wasting.proportion_exposed.1.sqlns.intervention_effect as modifier to child_wasting.proportion_exposed
2023-02-25 14:54:56.032 | DEBUG    | vivarium.framework.values:_register_value_producer:338 - Registering value pipeline sqlns.effect_size
2023-02-25 14:54:56.032 | DEBUG    | vivarium.framework.values:register_value_modifier:392 - Registering metrics.3.observer.metrics as modifier to metrics
2023-02-25 14:54:56.033 | DEBUG    | vivarium.framework.values:_register_value_producer:338 - Registering value pipeline metrics
>>> sim1.step()
2023-02-25 14:55:04.430 | DEBUG    | vivarium.framework.engine:step:198 - 2022-01-01 00:00:00
>>> sim2.step()
2023-02-25 14:55:09.652 | DEBUG    | vivarium.framework.engine:step:198 - 2022-01-01 00:00:00
>>>

This should be fairly straightforward to fix. Steps:

  1. Add a way for SimulationContext objects to generate a unique name
  2. Make logging a proper subsystem of vivarium and bind the emitted logs to the simulation context

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions