Skip to content

Commit efdb8d0

Browse files
committed
update short version
1 parent 80be21e commit efdb8d0

File tree

1 file changed

+6
-8
lines changed

1 file changed

+6
-8
lines changed

docs/src/maths/nondiff_points.md

Lines changed: 6 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,10 @@
11
# What to return for non-differentiable points
22
!!! info "What is the short version?"
3-
If the function is not differentiable due to e.g. a branch, like `abs`, your rule can reasonably claim the derivative at that point is the value from either branch, *or* any value in-between (e.g. for `abs` claiming 0 is a good idea).
4-
If it is not differentiable due to the primal not being defined on one side, you can set it to what ever you like.
5-
Your rule should claim a derivative that is *useful*.
6-
In calculus one learns that if the derivative as computed by approaching from the left,
7-
and the derivative one computes as approaching from the right are not equal then the derivative is not defined,
8-
and we say the function is not differentiable at that point.
9-
This is distinct from the notion captured by [`NoTangent`](@ref), which is that the tangent space itself is not defined: because in some sense the primal value can not be perturbed e.g. is is a discrete type.
3+
If the function is not-differentiable choose to return something useful rather than erroring.
4+
For a branch a function is not differentiable due to e.g. a branch, like `abs`, your rule can reasonably claim the derivative at that point is the value from either branch, *or* any value in-between.
5+
In particular for local optima (like in the case of `abs`) claiming the derivative is 0 is a good idea.
6+
Similarly, if derivative is from one side is not defined, or is not finite, return the derivative from the other side.
7+
Throwing an error, or returning `NaN` is generally the least useful option.
108

119
However, contrary to what calculus says most autodiff systems will return an answer for such functions.
1210
For example for: `abs_left(x) = (x <= 0) ? -x : x`, AD will say the derivative at `x=0` is `-1`.
@@ -137,4 +135,4 @@ These rough rules are:
137135
- If the derivative from one side is finite and the other isn't, say it is the derivative taken from finite side.
138136
- When derivative from each side is not equal, strongly consider reporting the average
139137

140-
Our goal as always, is to get a pragmatically useful result for everyone, which must by necessity also avoid a pathological result for anyone.
138+
Our goal as always, is to get a pragmatically useful result for everyone, which must by necessity also avoid a pathological result for anyone.

0 commit comments

Comments
 (0)