Grokfast exponential moving average Optimizer #176
Description
Motivation and description
Algorithm 2 in https://arxiv.org/pdf/2405.20233 should be a very easy implementation
Possible Implementation
No response
{{ message }}
Algorithm 2 in https://arxiv.org/pdf/2405.20233 should be a very easy implementation
No response