Skip to content

Commit 9f1966f

Browse files
improve ExpDecays docstring
1 parent 5f314ac commit 9f1966f

File tree

1 file changed

+13
-5
lines changed

1 file changed

+13
-5
lines changed

src/optimise/optimisers.jl

Lines changed: 13 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -569,6 +569,9 @@ for more general scheduling techniques.
569569
570570
# Examples
571571
572+
`InvDecay` is tipically composed with other optimizers
573+
as the last transformation of the gradient:
574+
572575
```julia
573576
# Inverse decay of the learning rate
574577
# with starting value 0.001 and decay coefficient 0.01.
@@ -604,12 +607,16 @@ a minimum of `clip`.
604607
two decay operations.
605608
- `clip`: Minimum value of learning rate.
606609
610+
611+
See also the [Scheduling Optimisers](@ref) section of the docs
612+
for more general scheduling techniques.
613+
607614
# Examples
608-
To apply exponential decay to an optimiser:
609-
```julia
610-
Optimiser(ExpDecay(..), Opt(..))
611615
612-
opt = Optimiser(ExpDecay(), ADAM())
616+
`ExpDecay` is tipically composed with other optimizers
617+
as the last transformation of the gradient:
618+
```julia
619+
opt = Optimiser(ADAM(), ExpDecay())
613620
```
614621
"""
615622
mutable struct ExpDecay <: AbstractOptimiser
@@ -620,7 +627,8 @@ mutable struct ExpDecay <: AbstractOptimiser
620627
current::IdDict
621628
end
622629

623-
ExpDecay(opt = 0.001, decay = 0.1, decay_step = 1000, clip = 1e-4) = ExpDecay(opt, decay, decay_step, clip, IdDict())
630+
ExpDecay(opt = 0.001, decay = 0.1, decay_step = 1000, clip = 1e-4) =
631+
ExpDecay(opt, decay, decay_step, clip, IdDict())
624632

625633
function apply!(o::ExpDecay, x, Δ)
626634
η, s, decay = o.eta, o.step, o.decay

0 commit comments

Comments
 (0)