Changing intial learning rate

[Khimer]

Hello!
I am trying to finetune the stt_ru_conformer_transducer_large model on new datasets using the adamw optimizer and scheduler CosineAnnealing. But when replacing the dataset, the initial learning rate is around 1e-10 or 1e-8 and grows slowly, which is very time consuming. I would like to be able to specify the initial learning rate at which training would start or restore its value from the last checkpoint, but unfortunately I only see the parameters model.cfg.optim.lr (set to 0.001) and model.cfg.optim. sched.min_lr (set as 1e-6), when changing which the initial learning rate still remains very small. Is it possible to specify the initial learning rate directly? Thank you!

[titu1994]

You can print out the optim config with print(model.cfg.optim) then change warmup_steps to a 0 or a small number like 5000. Probably the warmup steps is too much. Alzi check that the scheduler is actually CosineAnnealing - then 0.001 is used.

If it's Noam, then optim.lr acts as a multiplier - so your actual LR is being multiplied by 0.001