Skip to content

Commit 12c5502

Browse files
committed
tweak formatting
1 parent cd7516c commit 12c5502

File tree

1 file changed

+1
-6
lines changed

1 file changed

+1
-6
lines changed

docs/src/design/changing_the_primal.md

Lines changed: 1 addition & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,6 @@ It might be surprising to some AD authors, who might expect just a function that
66
In particularly, `rrule` allows you to _change_ how the primal result is computed.
77
We will illustrate in this document why being able to change the computation of the primal is crucial for efficient AD.
88

9-
109
!!! note "What about `frule`?"
1110
Discussion here is focused on on reverse mode and `rrule`.
1211
Similar concerns do apply to forward mode and `frule`.
@@ -15,9 +14,6 @@ We will illustrate in this document why being able to change the computation of
1514
In fact in forward mode there are even more opportunities to take advantage of sharing work between the primal and derivative computations.
1615
A particularly notable example is in efficiently calculating the pushforward of solving a differential equation via expanding the system of equations to also include the derivatives before solving it.
1716

18-
19-
20-
2117
## The Journey to `rrule`
2218

2319
Let's imagine a different system for rules, one that doesn't let you define the computation of the primal.
@@ -136,7 +132,6 @@ And it is faster to reuse the `exp(x)` in computing `σ(x)` and `σ(-x)`.
136132
How can we incorporate this insight into our system?
137133
We know we can compute both of these in the primal — because they only depend on `x` and not on `` — but there is nowhere to put them that is accessible both to the primal pass and the gradient pass code.
138134

139-
140135
What if we introduced some variable called `intermediates` that is also recorded onto the tape during the primal pass?
141136
We would need to be able to modify the primal pass to do this, so that we can actually put the data into the `intermediates`.
142137
So we will introduce a function: `augmented_primal`, that will return the primal output plus the `intermediates` that we want to reuse in the gradient pass.
@@ -475,7 +470,7 @@ We don't have this in ChainRules.jl yet, because Julia is missing some definitio
475470
I have been promised them for Julia v1.7 though.
476471
You can see what the code would look like in [PR #302](https://github.com/JuliaDiff/ChainRules.jl/pull/302).
477472

478-
# Conclusion
473+
## Conclusion
479474
This document has explained why [`rrule`](@ref) is the way it is.
480475
In particular it has highlighted why the primal computation is able to be changed from simply calling the function.
481476
Further, it has explained why `rrule` returns a closure for the pullback, rather than it being a separate function.

0 commit comments

Comments
 (0)