-
Notifications
You must be signed in to change notification settings - Fork 678
Open
Labels
better engineeringTasks which help improve eng productivity e.g. building tools, cleaning up code, writing docsTasks which help improve eng productivity e.g. building tools, cleaning up code, writing docsbugSomething isn't workingSomething isn't working
Description
Currently our WandBLogger has this code:
try:
output_config_fname = Path(
os.path.join(
config.checkpointer.checkpoint_dir,
"torchtune_config.yaml",
)
)
OmegaConf.save(config, output_config_fname)
log.info(f"Logging {output_config_fname} to W&B under Files")
self._wandb.save(
output_config_fname, base_path=output_config_fname.parent
)
except Exception as e:
log.warning(
f"Error saving {output_config_fname} to W&B.\nError: \n{e}."
"Don't worry the config will be logged the W&B workspace"
)
It is great that is saves the config, however, there are a few problems with it:
- This should be done for every config, not only WandBLogger. One option is to add this to every logger, but maybe this should be elsewhere, so it is not related to the logger.
- It saves it to the checkpoint_dir, which doesnt make sense. It should save the config to output_dir/...
- it does try/except, but if it fails to load output_config_fname, the except will also fail, because it tries to print "output_config_fname"
Metadata
Metadata
Assignees
Labels
better engineeringTasks which help improve eng productivity e.g. building tools, cleaning up code, writing docsTasks which help improve eng productivity e.g. building tools, cleaning up code, writing docsbugSomething isn't workingSomething isn't working