move NNlib's activation functions to their own page

Author: mcabbott

docs/make.jl

@@ -19,6 +19,7 @@ makedocs(
             "Regularisation" => "models/regularisation.md",
             "Advanced Model Building" => "models/advanced.md",
             "NNlib.jl" => "models/nnlib.md",
+            "Activation Functions" => "models/activation.md",
             "Functors.jl" => "models/functors.md",
          ],
          "Handling Data" => [

docs/src/models/activation.md

@@ -0,0 +1,39 @@
+
+# Activation Functions from NNlib.jl
+
+These non-linearities used between layers of your model are exported by the [NNlib](https://github.com/FluxML/NNlib.jl) package.
+
+Note that, unless otherwise stated, activation functions operate on scalars. To apply them to an array you can call `σ.(xs)`, `relu.(xs)` and so on. Alternatively, they can be passed to a layer like `Dense(784 => 1024, relu)` which will handle this broadcasting.
+
+```@docs
+celu
+elu
+gelu
+hardsigmoid
+sigmoid_fast
+hardtanh
+tanh_fast
+leakyrelu
+lisht
+logcosh
+logsigmoid
+mish
+relu
+relu6
+rrelu
+selu
+sigmoid
+softplus
+softshrink
+softsign
+swish
+hardswish
+tanhshrink
+trelu
+```
+
+Julia's `Base.Math` also provide `tanh`, which can be used as an activation function:
+
+```@docs
+tanh
+```

docs/src/models/nnlib.md

@@ -1,37 +1,6 @@
 # Neural Network primitives from NNlib.jl
 
-Flux re-exports all of the functions exported by the [NNlib](https://github.com/FluxML/NNlib.jl) package.
-
-## Activation Functions
-
-Non-linearities that go between layers of your model. Note that, unless otherwise stated, activation functions operate on scalars. To apply them to an array you can call `σ.(xs)`, `relu.(xs)` and so on.
-
-```@docs
-celu
-elu
-gelu
-hardsigmoid
-sigmoid_fast
-hardtanh
-tanh_fast
-leakyrelu
-lisht
-logcosh
-logsigmoid
-mish
-relu
-relu6
-rrelu
-selu
-sigmoid
-softplus
-softshrink
-softsign
-swish
-hardswish
-tanhshrink
-trelu
-```
+Flux re-exports all of the functions exported by the [NNlib](https://github.com/FluxML/NNlib.jl) package. This includes activation functions, described on the next page. Many of the functions on this page exist primarily as the internal implementation of Flux layer, but can also be used independently.
 
 ## Softmax