UserWarning: Detected call of lr_scheduler.step() before optimizer.step()

[mehadi92]

Getting this warning in nvcr.io/nvidia/nemo:23.03. Should I ignore it?

In PyTorch 1.1.0 and later, you should call them in the opposite order: `optimizer.step()` before `lr_scheduler.step()`.  Failure to do this will result in PyTorch skipping the first value of the learning rate schedule. See more details at https://pytorch.org/docs/stable/optim.html#how-to-adjust-learning-rate
      warnings.warn("Detected call of `lr_scheduler.step()` before `optimizer.step()`. "

[titu1994]

It can be ignored. It can happen sometime with DDP

[mehadi92]

I'm using ddp_find_unused_parameters_false. is it supposed to happen for this also?

[titu1994]

Yes, and that flag isn't really required btw. We just use ddp for strategy