Does "AdamW and CosineAnealing LR scheduler" always go to it's max LR before start decreasing the LR

[mehadi92]

Hi,
I'm checking this ASR Example CommonVoice Finetuning

Here you mention We use Learning Rate warmup when LR goes from 0 to maximum LR to stabilize initial phase of training

My question is does LR always (granted) go to the maximum (Set by us) LR value? or it's adaptive that is LR may start decreasing before it reaching to its maximum value?

Thanks

[titu1994]

Only Noam scheduler is adaptive. All other schedulers do reach the peak lr