@@ -29,7 +29,6 @@ for data in train_set
29
29
end
30
30
```
31
31
32
- It is important that every ` update! ` step receives a newly gradient computed gradient.
33
32
This loop can also be written using the function [ ` train! ` ] (@ref Flux.Train.train!),
34
33
but it's helpful to undersand the pieces first:
35
34
43
42
44
43
Fist recall from the section on [ taking gradients] (@ref man-taking-gradients) that
45
44
` Flux.gradient(f, a, b) ` always calls ` f(a, b) ` , and returns a tuple ` (∂f_∂a, ∂f_∂b) ` .
46
- In the code above, the function ` f ` is an anonymous function with one argument,
47
- created by the ` do ` block, hence ` grads ` is a tuple with one element.
45
+ In the code above, the function ` f ` passed to ` gradient ` is an anonymous function with
46
+ one argument, created by the ` do ` block, hence ` grads ` is a tuple with one element.
48
47
Instead of a ` do ` block, we could have written:
49
48
50
49
``` julia
@@ -58,6 +57,9 @@ structures are what Zygote calls "explicit" gradients.
58
57
It is important that the execution of the model takes place inside the call to ` gradient ` ,
59
58
in order for the influence of the model's parameters to be observed by Zygote.
60
59
60
+ It is also important that every ` update! ` step receives a newly gradient computed gradient,
61
+ as this will be change whenever the model's parameters are changed, and for each new data point.
62
+
61
63
!!! compat "Explicit vs implicit gradients"
62
64
Flux ≤ 0.13 used Zygote's "implicit" mode, in which ` gradient ` takes a zero-argument function.
63
65
It looks like this:
0 commit comments