error "Key not found" in loss computation when training on obs datasets #430
Description
What happened?
I get key errors when training on SEVIRI, NPP ATMS and Surface Combined due to a mismatch in the names.
The code removes all spaces and special characters, but the stream names contain spaces and special characters like "Surface Combined" or "NPP, ATMS".
I prepared a quick fix for the issue. opening the PR in a second.
What are the steps to reproduce the bug?
run with streams_mixed
Version
develop
Platform (OS and architecture)
ATOS
Relevant log output
Traceback (most recent call last):
File "/etc/ecmwf/nfs/dh2_home_a/ecm9336/evaluation/WeatherGenerator/src/weathergen/run_train.py", line 160, in train_with_args
trainer.run(cf)
File "/etc/ecmwf/nfs/dh2_home_a/ecm9336/evaluation/WeatherGenerator/src/weathergen/train/trainer.py", line 316, in run
self.train(epoch)
File "/etc/ecmwf/nfs/dh2_home_a/ecm9336/evaluation/WeatherGenerator/src/weathergen/train/trainer.py", line 573, in train
self.log(TRAIN)
File "/etc/ecmwf/nfs/dh2_home_a/ecm9336/evaluation/WeatherGenerator/src/weathergen/train/trainer.py", line 758, in log
self.train_logger.add_train(
File "/etc/ecmwf/nfs/dh2_home_a/ecm9336/evaluation/WeatherGenerator/src/weathergen/utils/train_logger.py", line 121, in add_train
loss = losses_all[st_name]
~~~~~~~~~~^^^^^^^^^
KeyError: 'NPPATMS'
> /etc/ecmwf/nfs/dh2_home_a/ecm9336/evaluation/WeatherGenerator/src/weathergen/utils/train_logger.py(121)add_train()
-> loss = losses_all[st_name]
(Pdb) Accompanying data
No response
Organisation
No response