From 5631d6cf23677055e26d5a9ef72bba1804f58d74 Mon Sep 17 00:00:00 2001 From: Michael Abbott <32575566+mcabbott@users.noreply.github.com> Date: Thu, 10 Nov 2022 11:23:43 -0500 Subject: [PATCH 01/15] remove train from quickstart example --- README.md | 6 ++--- docs/src/models/quickstart.md | 46 ++++++++++++++++++----------------- 2 files changed, 27 insertions(+), 25 deletions(-) diff --git a/README.md b/README.md index 1dec5940e7..39eec75448 100644 --- a/README.md +++ b/README.md @@ -18,12 +18,12 @@ Flux is an elegant approach to machine learning. It's a 100% pure-Julia stack, and provides lightweight abstractions on top of Julia's native GPU and AD support. Flux makes the easy things easy while remaining fully hackable. -Works best with [Julia 1.8](https://julialang.org/downloads/) or later. Here's a simple example to try it out: +Works best with [Julia 1.8](https://julialang.org/downloads/) or later. Here's a very short example to try it out: ```julia using Flux # should install everything for you, including CUDA x = hcat(digits.(0:3, base=2, pad=2)...) |> gpu # let's solve the XOR problem! -y = Flux.onehotbatch(xor.(eachrow(x)...), 0:1) |> gpu +y = Flux.onehotbatch(xor.(eachrow(cpu(x))...), 0:1) |> gpu data = ((Float32.(x), y) for _ in 1:100) # an iterator making Tuples model = Chain(Dense(2 => 3, sigmoid), BatchNorm(3), Dense(3 => 2)) |> gpu @@ -35,6 +35,6 @@ Flux.train!(mloss, Flux.params(model), data, optim) # updates model & optim all((softmax(model(x)) .> 0.5) .== y) # usually 100% accuracy. ``` -See the [documentation](https://fluxml.github.io/Flux.jl/) for details, or the [model zoo](https://github.com/FluxML/model-zoo/) for examples. Ask questions on the [Julia discourse](https://discourse.julialang.org/) or [slack](https://discourse.julialang.org/t/announcing-a-julia-slack/4866). +The [quickstart page](https://fluxml.ai/Flux.jl/stable/models/quickstart/) has a longer version. See the [documentation](https://fluxml.github.io/Flux.jl/) for details, or the [model zoo](https://github.com/FluxML/model-zoo/) for examples. Ask questions on the [Julia discourse](https://discourse.julialang.org/) or [slack](https://discourse.julialang.org/t/announcing-a-julia-slack/4866). If you use Flux in your research, please [cite](CITATION.bib) our work. diff --git a/docs/src/models/quickstart.md b/docs/src/models/quickstart.md index 28685176e9..01d4292ad8 100644 --- a/docs/src/models/quickstart.md +++ b/docs/src/models/quickstart.md @@ -18,26 +18,36 @@ model = Chain(Dense(2 => 3, tanh), BatchNorm(3), Dense(3 => 2), softmax) # The model encapsulates parameters, randomly initialised. Its initial output is: out1 = model(noisy) # 2×1000 Matrix{Float32} -# To train the model, we use batches of 64 samples: -mat = Flux.onehotbatch(truth, [true, false]) # 2×1000 OneHotMatrix -data = Flux.DataLoader((noisy, mat), batchsize=64, shuffle=true); -first(data) .|> summary # ("2×64 Matrix{Float32}", "2×64 Matrix{Bool}") +# To train the model, we use batches of 64 samples, and one-hot encoding: +target = Flux.onehotbatch(truth, [true, false]) # 2×1000 OneHotMatrix +loader = Flux.DataLoader((noisy, target), batchsize=64, shuffle=true); +first(loader) .|> summary # ("2×64 Matrix{Float32}", "2×64 Matrix{Bool}") pars = Flux.params(model) # contains references to arrays in model opt = Flux.Adam(0.01) # will store optimiser momentum, etc. # Training loop, using the whole data set 1000 times: for epoch in 1:1_000 - Flux.train!(pars, data, opt) do x, y - # First argument of train! is a loss function, here defined by a `do` block. - # This gets x and y, each a 2×64 Matrix, from data, and compares: - Flux.crossentropy(model(x), y) + losses = [] + for (x, y) in loader + loss, grad = Flux.withgradient(pars) do + # Evaluate model and loss inside gradient context: + y_hat = model(x) + Flux.crossentropy(y_hat, y) # could use just sum(abs2, y_hat .- y) + end + # Use the gradient to update the model's parameters (and momentum): + Flux.update!(opt, pars, grad) + # Logging code, outside gradient context: + push!(losses, loss) + end + if isinteger(log2(epoch)) + println("after epoch $epoch, loss is ", mean(losses)) end end -pars # has changed! +pars # parameters, momenta and output have all changed opt -out2 = model(noisy) +out2 = model(noisy) # first row is prob. of true, second row p(false) mean((out2[1,:] .> 0.5) .== truth) # accuracy 94% so far! ``` @@ -62,18 +72,10 @@ Since then things have developed a little. Some things to notice in this example are: -* The batch dimension of data is always the last one. Thus a `2×1000 Matrix` is a thousand observations, each a column of length 2. - -* The `model` can be called like a function, `y = model(x)`. It encapsulates the parameters (and state). - -* But the model does not contain the loss function, nor the optimisation rule. Instead the [`Adam()`](@ref Flux.Adam) object stores between iterations the momenta it needs. - -* The function [`train!`](@ref Flux.train!) likes data as an iterator generating `Tuple`s, here produced by [`DataLoader`](@ref). This mutates both the `model` and the optimiser state inside `opt`. - -There are other ways to train Flux models, for more control than `train!` provides: +* The batch dimension of data is always the last one. Thus a `2×1000 Matrix` is a thousand observations, each a column of length 2. Flux defaults to `Float32`, but most of Julia to `Float64`. -* Within Flux, you can easily write a training loop, calling [`gradient`](@ref) and [`update!`](@ref Flux.update!). +* The `model` can be called like a function, `y = model(x)`. Each layer like [`Dense`](@ref ) is an ordinary `struct`, which encapsulates some arrays of parameters (and possibly other state, as for [`BatchNorm`](@ref)). -* For a lower-level way, see the package [Optimisers.jl](https://github.com/FluxML/Optimisers.jl). +* But the model does not contain the loss function, nor the optimisation rule. The [`Adam`](@ref Flux.Adam) object stores between iterations the momenta it needs. And [`Flux.crossentropy`](@ref Flux.Losses.crossentropy) is an ordinary function. -* For higher-level ways, see [FluxTraining.jl](https://github.com/FluxML/FluxTraining.jl) and [FastAI.jl](https://github.com/FluxML/FastAI.jl). +* Instead of calling [`gradient`](@ref Zygote.gradient) and [`update!`](@ref Flux.update!) separately, there is a convenience function [`train!`](@ref Flux.train!) which we could use. However, to do anything extra (like logging the loss) an explicit loop is usually clearest. From adb1f8b442d8f6cbcc7c687dc993823fd2cf521f Mon Sep 17 00:00:00 2001 From: Michael Abbott <32575566+mcabbott@users.noreply.github.com> Date: Fri, 11 Nov 2022 00:48:22 -0500 Subject: [PATCH 02/15] fixes & suggestions --- docs/src/models/quickstart.md | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-) diff --git a/docs/src/models/quickstart.md b/docs/src/models/quickstart.md index 01d4292ad8..006641c976 100644 --- a/docs/src/models/quickstart.md +++ b/docs/src/models/quickstart.md @@ -10,10 +10,14 @@ using Flux, Statistics # Generate some data for the XOR problem: vectors of length 2, as columns of a matrix: noisy = rand(Float32, 2, 1000) # 2×1000 Matrix{Float32} -truth = map(col -> xor(col...), eachcol(noisy .> 0.5)) # 1000-element Vector{Bool} +truth = [xor(col[1]>0.5, col[2]>0.5) for col in eachcol(noisy)] # 1000-element Vector{Bool} # Define our model, a multi-layer perceptron with one hidden layer of size 3: -model = Chain(Dense(2 => 3, tanh), BatchNorm(3), Dense(3 => 2), softmax) +model = Chain( + Dense(2 => 3, tanh), # activation function inside... + BatchNorm(3), + Dense(3 => 2), + softmax) # ... but softmax outside a layer. # The model encapsulates parameters, randomly initialised. Its initial output is: out1 = model(noisy) # 2×1000 Matrix{Float32} @@ -21,7 +25,7 @@ out1 = model(noisy) # 2×1000 Matr # To train the model, we use batches of 64 samples, and one-hot encoding: target = Flux.onehotbatch(truth, [true, false]) # 2×1000 OneHotMatrix loader = Flux.DataLoader((noisy, target), batchsize=64, shuffle=true); -first(loader) .|> summary # ("2×64 Matrix{Float32}", "2×64 Matrix{Bool}") +# 16-element DataLoader with first element: (2×64 Matrix{Float32}, 2×64 OneHotMatrix) pars = Flux.params(model) # contains references to arrays in model opt = Flux.Adam(0.01) # will store optimiser momentum, etc. @@ -74,8 +78,10 @@ Some things to notice in this example are: * The batch dimension of data is always the last one. Thus a `2×1000 Matrix` is a thousand observations, each a column of length 2. Flux defaults to `Float32`, but most of Julia to `Float64`. -* The `model` can be called like a function, `y = model(x)`. Each layer like [`Dense`](@ref ) is an ordinary `struct`, which encapsulates some arrays of parameters (and possibly other state, as for [`BatchNorm`](@ref)). +* The `model` can be called like a function, `y = model(x)`. Each layer like [`Dense`](@ref Flux.Dense) is an ordinary `struct`, which encapsulates some arrays of parameters (and possibly other state, as for [`BatchNorm`](@ref Flux.BatchNorm)). * But the model does not contain the loss function, nor the optimisation rule. The [`Adam`](@ref Flux.Adam) object stores between iterations the momenta it needs. And [`Flux.crossentropy`](@ref Flux.Losses.crossentropy) is an ordinary function. -* Instead of calling [`gradient`](@ref Zygote.gradient) and [`update!`](@ref Flux.update!) separately, there is a convenience function [`train!`](@ref Flux.train!) which we could use. However, to do anything extra (like logging the loss) an explicit loop is usually clearest. +* Instead of calling [`gradient`](@ref Zygote.gradient) and [`update!`](@ref Flux.update!) separately, there is a convenience function [`train!`](@ref Flux.train!) which could replace the `for (x, y) in loader` loop. However, to do anything extra (like logging the loss) an explicit loop is usually clearest. + +* The `do` block creates an anonymous function, as the first argument of `gradient`. Anything executed within this is differentiated. From 221d435aeb02621b74883663491035734e90e4ec Mon Sep 17 00:00:00 2001 From: Michael Abbott <32575566+mcabbott@users.noreply.github.com> Date: Wed, 16 Nov 2022 08:29:22 -0500 Subject: [PATCH 03/15] better bullet points --- docs/src/models/quickstart.md | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/docs/src/models/quickstart.md b/docs/src/models/quickstart.md index 006641c976..f2874b4bae 100644 --- a/docs/src/models/quickstart.md +++ b/docs/src/models/quickstart.md @@ -76,12 +76,14 @@ Since then things have developed a little. Some things to notice in this example are: -* The batch dimension of data is always the last one. Thus a `2×1000 Matrix` is a thousand observations, each a column of length 2. Flux defaults to `Float32`, but most of Julia to `Float64`. +* The batch dimension of data is always the last one. Thus a `2×1000 Matrix` is a thousand observations, each a column of length 2. + +* Flux defaults to `Float32`, but most of Julia to `Float64`. * The `model` can be called like a function, `y = model(x)`. Each layer like [`Dense`](@ref Flux.Dense) is an ordinary `struct`, which encapsulates some arrays of parameters (and possibly other state, as for [`BatchNorm`](@ref Flux.BatchNorm)). * But the model does not contain the loss function, nor the optimisation rule. The [`Adam`](@ref Flux.Adam) object stores between iterations the momenta it needs. And [`Flux.crossentropy`](@ref Flux.Losses.crossentropy) is an ordinary function. -* Instead of calling [`gradient`](@ref Zygote.gradient) and [`update!`](@ref Flux.update!) separately, there is a convenience function [`train!`](@ref Flux.train!) which could replace the `for (x, y) in loader` loop. However, to do anything extra (like logging the loss) an explicit loop is usually clearest. - * The `do` block creates an anonymous function, as the first argument of `gradient`. Anything executed within this is differentiated. + +Instead of calling [`gradient`](@ref Zygote.gradient) and [`update!`](@ref Flux.update!) separately, there is a convenience function [`train!`](@ref Flux.train!) which could replace the `for (x, y) in loader` loop. However, to do anything extra (like logging the loss) an explicit loop is usually clearest. From 0faaa25b08e6bf24d63f34480efdd778906f92ce Mon Sep 17 00:00:00 2001 From: Michael Abbott <32575566+mcabbott@users.noreply.github.com> Date: Wed, 16 Nov 2022 08:35:18 -0500 Subject: [PATCH 04/15] dump train! and gpu from the readme too --- README.md | 20 +++++++++++++------- 1 file changed, 13 insertions(+), 7 deletions(-) diff --git a/README.md b/README.md index 39eec75448..8af0618db4 100644 --- a/README.md +++ b/README.md @@ -20,17 +20,19 @@ Flux is an elegant approach to machine learning. It's a 100% pure-Julia stack, a Works best with [Julia 1.8](https://julialang.org/downloads/) or later. Here's a very short example to try it out: ```julia -using Flux # should install everything for you, including CUDA +using Flux, Zygote # should install everything for you, including CUDA -x = hcat(digits.(0:3, base=2, pad=2)...) |> gpu # let's solve the XOR problem! -y = Flux.onehotbatch(xor.(eachrow(cpu(x))...), 0:1) |> gpu -data = ((Float32.(x), y) for _ in 1:100) # an iterator making Tuples +x = hcat(digits.(0:3, base=2, pad=2)...) # data for the XOR problem +y = Flux.onehotbatch(xor.(eachrow(x)...), 0:1) -model = Chain(Dense(2 => 3, sigmoid), BatchNorm(3), Dense(3 => 2)) |> gpu +model = Chain(Dense(2 => 3, sigmoid), BatchNorm(3), Dense(3 => 2)) +pars = Flux.params(model) # a dictionary of arrays in model optim = Adam(0.1, (0.7, 0.95)) -mloss(x, y) = Flux.logitcrossentropy(model(x), y) # closes over model -Flux.train!(mloss, Flux.params(model), data, optim) # updates model & optim +for _ in 1:100 + grad = gradient(() -> Flux.logitcrossentropy(model(x), y), pars) + Flux.update!(optim, pars, grad) # this changes model & optim +end all((softmax(model(x)) .> 0.5) .== y) # usually 100% accuracy. ``` @@ -38,3 +40,7 @@ all((softmax(model(x)) .> 0.5) .== y) # usually 100% accuracy. The [quickstart page](https://fluxml.ai/Flux.jl/stable/models/quickstart/) has a longer version. See the [documentation](https://fluxml.github.io/Flux.jl/) for details, or the [model zoo](https://github.com/FluxML/model-zoo/) for examples. Ask questions on the [Julia discourse](https://discourse.julialang.org/) or [slack](https://discourse.julialang.org/t/announcing-a-julia-slack/4866). If you use Flux in your research, please [cite](CITATION.bib) our work. + + + + From 27e1600a1fc886524951aec49cc0df7c39496ba4 Mon Sep 17 00:00:00 2001 From: Michael Abbott <32575566+mcabbott@users.noreply.github.com> Date: Wed, 16 Nov 2022 18:00:09 -0500 Subject: [PATCH 05/15] remove a few comments --- docs/src/models/quickstart.md | 14 ++++++-------- 1 file changed, 6 insertions(+), 8 deletions(-) diff --git a/docs/src/models/quickstart.md b/docs/src/models/quickstart.md index f2874b4bae..a83b234789 100644 --- a/docs/src/models/quickstart.md +++ b/docs/src/models/quickstart.md @@ -6,7 +6,7 @@ If you haven't, then you might prefer the [Fitting a Straight Line](overview.md) ```julia # With Julia 1.7+, this will prompt if neccessary to install everything, including CUDA: -using Flux, Statistics +using Flux, Zygote, Statistics # Generate some data for the XOR problem: vectors of length 2, as columns of a matrix: noisy = rand(Float32, 2, 1000) # 2×1000 Matrix{Float32} @@ -14,10 +14,10 @@ truth = [xor(col[1]>0.5, col[2]>0.5) for col in eachcol(noisy)] # 1000-element # Define our model, a multi-layer perceptron with one hidden layer of size 3: model = Chain( - Dense(2 => 3, tanh), # activation function inside... + Dense(2 => 3, tanh), # activation function inside layer BatchNorm(3), Dense(3 => 2), - softmax) # ... but softmax outside a layer. + softmax) # The model encapsulates parameters, randomly initialised. Its initial output is: out1 = model(noisy) # 2×1000 Matrix{Float32} @@ -34,15 +34,13 @@ opt = Flux.Adam(0.01) # will store optimiser momentum, etc. for epoch in 1:1_000 losses = [] for (x, y) in loader - loss, grad = Flux.withgradient(pars) do + loss, grad = Zygote.withgradient(pars) do # Evaluate model and loss inside gradient context: y_hat = model(x) - Flux.crossentropy(y_hat, y) # could use just sum(abs2, y_hat .- y) + Flux.crossentropy(y_hat, y) end - # Use the gradient to update the model's parameters (and momentum): Flux.update!(opt, pars, grad) - # Logging code, outside gradient context: - push!(losses, loss) + push!(losses, loss) # logging, outside gradient context end if isinteger(log2(epoch)) println("after epoch $epoch, loss is ", mean(losses)) From 0cb65e30e20d2fd553dbb10bb524782f5c59a13a Mon Sep 17 00:00:00 2001 From: Michael Abbott <32575566+mcabbott@users.noreply.github.com> Date: Wed, 16 Nov 2022 21:19:28 -0500 Subject: [PATCH 06/15] rm mention of Zygote --- README.md | 2 +- docs/src/models/quickstart.md | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 8af0618db4..25814d4801 100644 --- a/README.md +++ b/README.md @@ -20,7 +20,7 @@ Flux is an elegant approach to machine learning. It's a 100% pure-Julia stack, a Works best with [Julia 1.8](https://julialang.org/downloads/) or later. Here's a very short example to try it out: ```julia -using Flux, Zygote # should install everything for you, including CUDA +using Flux # should install everything for you, including CUDA x = hcat(digits.(0:3, base=2, pad=2)...) # data for the XOR problem y = Flux.onehotbatch(xor.(eachrow(x)...), 0:1) diff --git a/docs/src/models/quickstart.md b/docs/src/models/quickstart.md index a83b234789..1bd1ea9b5e 100644 --- a/docs/src/models/quickstart.md +++ b/docs/src/models/quickstart.md @@ -6,7 +6,7 @@ If you haven't, then you might prefer the [Fitting a Straight Line](overview.md) ```julia # With Julia 1.7+, this will prompt if neccessary to install everything, including CUDA: -using Flux, Zygote, Statistics +using Flux, Statistics # Generate some data for the XOR problem: vectors of length 2, as columns of a matrix: noisy = rand(Float32, 2, 1000) # 2×1000 Matrix{Float32} @@ -34,7 +34,7 @@ opt = Flux.Adam(0.01) # will store optimiser momentum, etc. for epoch in 1:1_000 losses = [] for (x, y) in loader - loss, grad = Zygote.withgradient(pars) do + loss, grad = withgradient(pars) do # Evaluate model and loss inside gradient context: y_hat = model(x) Flux.crossentropy(y_hat, y) From 4ba8f7456fb146caa1223076cea5535a4cc40a32 Mon Sep 17 00:00:00 2001 From: Michael Abbott <32575566+mcabbott@users.noreply.github.com> Date: Sun, 20 Nov 2022 15:38:09 -0500 Subject: [PATCH 07/15] maybe we should have a much simpler readme example --- README.md | 22 ++++++++++------------ 1 file changed, 10 insertions(+), 12 deletions(-) diff --git a/README.md b/README.md index 25814d4801..07596cfce7 100644 --- a/README.md +++ b/README.md @@ -20,24 +20,22 @@ Flux is an elegant approach to machine learning. It's a 100% pure-Julia stack, a Works best with [Julia 1.8](https://julialang.org/downloads/) or later. Here's a very short example to try it out: ```julia -using Flux # should install everything for you, including CUDA +using Flux, Plots +data = [([x], x-cbrt(x)) for x in range(-2, 2, 100)] -x = hcat(digits.(0:3, base=2, pad=2)...) # data for the XOR problem -y = Flux.onehotbatch(xor.(eachrow(x)...), 0:1) +model = Chain(Dense(1 => 10, tanh), Dense(10 => 1), only) -model = Chain(Dense(2 => 3, sigmoid), BatchNorm(3), Dense(3 => 2)) -pars = Flux.params(model) # a dictionary of arrays in model -optim = Adam(0.1, (0.7, 0.95)) - -for _ in 1:100 - grad = gradient(() -> Flux.logitcrossentropy(model(x), y), pars) - Flux.update!(optim, pars, grad) # this changes model & optim +loss(x,y) = abs2(model(x) - y) +optim = Flux.Adam() +for epoch in 1:1000 + Flux.train!(loss, Flux.params(model), data, optim) end -all((softmax(model(x)) .> 0.5) .== y) # usually 100% accuracy. +plot(x -> x-cbrt(x), -2, 2, legend=false) +scatter!(-2:0.1:2, [model([x]) for x in -2:0.1:2]) ``` -The [quickstart page](https://fluxml.ai/Flux.jl/stable/models/quickstart/) has a longer version. See the [documentation](https://fluxml.github.io/Flux.jl/) for details, or the [model zoo](https://github.com/FluxML/model-zoo/) for examples. Ask questions on the [Julia discourse](https://discourse.julialang.org/) or [slack](https://discourse.julialang.org/t/announcing-a-julia-slack/4866). +The [quickstart page](https://fluxml.ai/Flux.jl/stable/models/quickstart/) has a longer example. See the [documentation](https://fluxml.github.io/Flux.jl/) for details, or the [model zoo](https://github.com/FluxML/model-zoo/) for examples. Ask questions on the [Julia discourse](https://discourse.julialang.org/) or [slack](https://discourse.julialang.org/t/announcing-a-julia-slack/4866). If you use Flux in your research, please [cite](CITATION.bib) our work. From 467c43239faac1dca63110b27032f569d49600ee Mon Sep 17 00:00:00 2001 From: Michael Abbott <32575566+mcabbott@users.noreply.github.com> Date: Sun, 20 Nov 2022 17:31:11 -0500 Subject: [PATCH 08/15] tweaks --- docs/src/models/quickstart.md | 15 +++++++++++---- 1 file changed, 11 insertions(+), 4 deletions(-) diff --git a/docs/src/models/quickstart.md b/docs/src/models/quickstart.md index 1bd1ea9b5e..6eb313c7bd 100644 --- a/docs/src/models/quickstart.md +++ b/docs/src/models/quickstart.md @@ -74,9 +74,7 @@ Since then things have developed a little. Some things to notice in this example are: -* The batch dimension of data is always the last one. Thus a `2×1000 Matrix` is a thousand observations, each a column of length 2. - -* Flux defaults to `Float32`, but most of Julia to `Float64`. +* The batch dimension of data is always the last one. Thus a `2×1000 Matrix` is a thousand observations, each a column of length 2. Flux defaults to `Float32`, but most of Julia to `Float64`. * The `model` can be called like a function, `y = model(x)`. Each layer like [`Dense`](@ref Flux.Dense) is an ordinary `struct`, which encapsulates some arrays of parameters (and possibly other state, as for [`BatchNorm`](@ref Flux.BatchNorm)). @@ -84,4 +82,13 @@ Some things to notice in this example are: * The `do` block creates an anonymous function, as the first argument of `gradient`. Anything executed within this is differentiated. -Instead of calling [`gradient`](@ref Zygote.gradient) and [`update!`](@ref Flux.update!) separately, there is a convenience function [`train!`](@ref Flux.train!) which could replace the `for (x, y) in loader` loop. However, to do anything extra (like logging the loss) an explicit loop is usually clearest. +Instead of calling [`gradient`](@ref Zygote.gradient) and [`update!`](@ref Flux.update!) separately, there is a convenience function [`train!`](@ref Flux.train!). If we didn't want anything extra (like logging the loss), we could replace the training loop with the following: + +````julia +for epoch in 1:1_000 + train!(pars, loader, opt) do x, y + y_hat = model(x) + Flux.crossentropy(y_hat, y) + end +end +``` From 881280b1bdf2b85708ff90610e7e75c7df04c306 Mon Sep 17 00:00:00 2001 From: Michael Abbott <32575566+mcabbott@users.noreply.github.com> Date: Thu, 24 Nov 2022 21:42:47 -0500 Subject: [PATCH 09/15] no more cbrt, no more abs2 --- README.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index 07596cfce7..757b130e40 100644 --- a/README.md +++ b/README.md @@ -21,17 +21,17 @@ Flux is an elegant approach to machine learning. It's a 100% pure-Julia stack, a Works best with [Julia 1.8](https://julialang.org/downloads/) or later. Here's a very short example to try it out: ```julia using Flux, Plots -data = [([x], x-cbrt(x)) for x in range(-2, 2, 100)] +data = [([x], 2x-x^3) for x in -2:0.1f0:2] -model = Chain(Dense(1 => 10, tanh), Dense(10 => 1), only) +model = Chain(Dense(1 => 23, tanh), Dense(23 => 1, bias=false), only) -loss(x,y) = abs2(model(x) - y) +loss(x,y) = (model(x) - y)^2 optim = Flux.Adam() for epoch in 1:1000 Flux.train!(loss, Flux.params(model), data, optim) end -plot(x -> x-cbrt(x), -2, 2, legend=false) +plot(x -> 2x-x^3, -2, 2, legend=false) scatter!(-2:0.1:2, [model([x]) for x in -2:0.1:2]) ``` From ed6470572409c91bb0678192e27e8c93a2acfb4e Mon Sep 17 00:00:00 2001 From: Michael Abbott <32575566+mcabbott@users.noreply.github.com> Date: Sat, 26 Nov 2022 14:05:37 -0500 Subject: [PATCH 10/15] remove controversial println code, and make it shorter --- docs/src/models/quickstart.md | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/docs/src/models/quickstart.md b/docs/src/models/quickstart.md index 6eb313c7bd..aef7863bc9 100644 --- a/docs/src/models/quickstart.md +++ b/docs/src/models/quickstart.md @@ -31,8 +31,8 @@ pars = Flux.params(model) # contains references to arrays in model opt = Flux.Adam(0.01) # will store optimiser momentum, etc. # Training loop, using the whole data set 1000 times: +losses = [] for epoch in 1:1_000 - losses = [] for (x, y) in loader loss, grad = withgradient(pars) do # Evaluate model and loss inside gradient context: @@ -42,9 +42,6 @@ for epoch in 1:1_000 Flux.update!(opt, pars, grad) push!(losses, loss) # logging, outside gradient context end - if isinteger(log2(epoch)) - println("after epoch $epoch, loss is ", mean(losses)) - end end pars # parameters, momenta and output have all changed From 739197d664bfe58cb8b6f9efb0d721970488b621 Mon Sep 17 00:00:00 2001 From: Michael Abbott <32575566+mcabbott@users.noreply.github.com> Date: Sat, 26 Nov 2022 14:08:03 -0500 Subject: [PATCH 11/15] fix some fences --- docs/src/models/quickstart.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/src/models/quickstart.md b/docs/src/models/quickstart.md index aef7863bc9..efdfde5f1e 100644 --- a/docs/src/models/quickstart.md +++ b/docs/src/models/quickstart.md @@ -53,7 +53,7 @@ mean((out2[1,:] .> 0.5) .== truth) # accuracy 94% so far! ![](../assets/oneminute.png) -``` +```julia using Plots # to draw the above figure p_true = scatter(noisy[1,:], noisy[2,:], zcolor=truth, title="True classification", legend=false) @@ -67,7 +67,7 @@ This XOR ("exclusive or") problem is a variant of the famous one which drove Min Since then things have developed a little. -## Features of Note +## Features to Note Some things to notice in this example are: @@ -81,7 +81,7 @@ Some things to notice in this example are: Instead of calling [`gradient`](@ref Zygote.gradient) and [`update!`](@ref Flux.update!) separately, there is a convenience function [`train!`](@ref Flux.train!). If we didn't want anything extra (like logging the loss), we could replace the training loop with the following: -````julia +```julia for epoch in 1:1_000 train!(pars, loader, opt) do x, y y_hat = model(x) From c0994c77fc211e52415ed74eca4ca806e3382a9e Mon Sep 17 00:00:00 2001 From: Michael Abbott <32575566+mcabbott@users.noreply.github.com> Date: Sat, 26 Nov 2022 14:34:47 -0500 Subject: [PATCH 12/15] maybe this example should run on the GPU, since it easily can, even though this is slower --- docs/src/models/quickstart.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/docs/src/models/quickstart.md b/docs/src/models/quickstart.md index efdfde5f1e..a5d5bb9454 100644 --- a/docs/src/models/quickstart.md +++ b/docs/src/models/quickstart.md @@ -17,14 +17,14 @@ model = Chain( Dense(2 => 3, tanh), # activation function inside layer BatchNorm(3), Dense(3 => 2), - softmax) + softmax) |> gpu # move model to GPU, if available # The model encapsulates parameters, randomly initialised. Its initial output is: -out1 = model(noisy) # 2×1000 Matrix{Float32} +out1 = model(noisy |> gpu) |> cpu # 2×1000 Matrix{Float32} # To train the model, we use batches of 64 samples, and one-hot encoding: target = Flux.onehotbatch(truth, [true, false]) # 2×1000 OneHotMatrix -loader = Flux.DataLoader((noisy, target), batchsize=64, shuffle=true); +loader = Flux.DataLoader((noisy, target) |> gpu, batchsize=64, shuffle=true); # 16-element DataLoader with first element: (2×64 Matrix{Float32}, 2×64 OneHotMatrix) pars = Flux.params(model) # contains references to arrays in model @@ -34,7 +34,7 @@ opt = Flux.Adam(0.01) # will store optimiser momentum, etc. losses = [] for epoch in 1:1_000 for (x, y) in loader - loss, grad = withgradient(pars) do + loss, grad = Flux.withgradient(pars) do # Evaluate model and loss inside gradient context: y_hat = model(x) Flux.crossentropy(y_hat, y) @@ -46,7 +46,7 @@ end pars # parameters, momenta and output have all changed opt -out2 = model(noisy) # first row is prob. of true, second row p(false) +out2 = model(noisy |> gpu) |> cpu # first row is prob. of true, second row p(false) mean((out2[1,:] .> 0.5) .== truth) # accuracy 94% so far! ``` From cd33b1ae7e576e83246d5f76ca35aa540dda69f8 Mon Sep 17 00:00:00 2001 From: Michael Abbott <32575566+mcabbott@users.noreply.github.com> Date: Sat, 26 Nov 2022 14:35:23 -0500 Subject: [PATCH 13/15] let's replace explicit printing with showprogress macro, it's pretty and doesn't waste lines --- docs/src/models/quickstart.md | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/docs/src/models/quickstart.md b/docs/src/models/quickstart.md index a5d5bb9454..ec26705aa1 100644 --- a/docs/src/models/quickstart.md +++ b/docs/src/models/quickstart.md @@ -6,7 +6,7 @@ If you haven't, then you might prefer the [Fitting a Straight Line](overview.md) ```julia # With Julia 1.7+, this will prompt if neccessary to install everything, including CUDA: -using Flux, Statistics +using Flux, Statistics, ProgressMeter # Generate some data for the XOR problem: vectors of length 2, as columns of a matrix: noisy = rand(Float32, 2, 1000) # 2×1000 Matrix{Float32} @@ -32,7 +32,7 @@ opt = Flux.Adam(0.01) # will store optimiser momentum, etc. # Training loop, using the whole data set 1000 times: losses = [] -for epoch in 1:1_000 +@showprogress for epoch in 1:1_000 for (x, y) in loader loss, grad = Flux.withgradient(pars) do # Evaluate model and loss inside gradient context: @@ -63,6 +63,14 @@ p_done = scatter(noisy[1,:], noisy[2,:], zcolor=out2[1,:], title="Trained networ plot(p_true, p_raw, p_done, layout=(1,3), size=(1000,330)) ``` +Here's the loss during training: + +```julia +plot(losses; xaxis=(:log10, "iteration"), yaxis="loss", label="per batch") +n = length(loader) +plot!(n:n:length(losses), mean.(Iterators.partition(losses, n)), label="epoch mean") +``` + This XOR ("exclusive or") problem is a variant of the famous one which drove Minsky and Papert to invent deep neural networks in 1969. For small values of "deep" -- this has one hidden layer, while earlier perceptrons had none. (What they call a hidden layer, Flux calls the output of the first layer, `model[1](noisy)`.) Since then things have developed a little. From 9a2abbee054350a9d9999fe39642565f56f7d1bf Mon Sep 17 00:00:00 2001 From: Michael Abbott <32575566+mcabbott@users.noreply.github.com> Date: Sat, 26 Nov 2022 15:34:33 -0500 Subject: [PATCH 14/15] add graph of the loss, since we log it? also move to a folder. --- README.md | 8 ++------ docs/src/assets/quickstart/loss.png | Bin 0 -> 62443 bytes docs/src/assets/{ => quickstart}/oneminute.png | Bin docs/src/models/quickstart.md | 12 +++++++++--- 4 files changed, 11 insertions(+), 9 deletions(-) create mode 100644 docs/src/assets/quickstart/loss.png rename docs/src/assets/{ => quickstart}/oneminute.png (100%) diff --git a/README.md b/README.md index 757b130e40..b3dda36a0f 100644 --- a/README.md +++ b/README.md @@ -25,10 +25,10 @@ data = [([x], 2x-x^3) for x in -2:0.1f0:2] model = Chain(Dense(1 => 23, tanh), Dense(23 => 1, bias=false), only) -loss(x,y) = (model(x) - y)^2 +mloss(x,y) = (model(x) - y)^2 optim = Flux.Adam() for epoch in 1:1000 - Flux.train!(loss, Flux.params(model), data, optim) + Flux.train!(mloss, Flux.params(model), data, optim) end plot(x -> 2x-x^3, -2, 2, legend=false) @@ -38,7 +38,3 @@ scatter!(-2:0.1:2, [model([x]) for x in -2:0.1:2]) The [quickstart page](https://fluxml.ai/Flux.jl/stable/models/quickstart/) has a longer example. See the [documentation](https://fluxml.github.io/Flux.jl/) for details, or the [model zoo](https://github.com/FluxML/model-zoo/) for examples. Ask questions on the [Julia discourse](https://discourse.julialang.org/) or [slack](https://discourse.julialang.org/t/announcing-a-julia-slack/4866). If you use Flux in your research, please [cite](CITATION.bib) our work. - - - - diff --git a/docs/src/assets/quickstart/loss.png b/docs/src/assets/quickstart/loss.png new file mode 100644 index 0000000000000000000000000000000000000000..8cfa5523d232a0bebd071d85b8a70f30926d14e4 GIT binary patch literal 62443 zcmd43byQX17dD6@pdbPQ(x4aV?goKNcXvp4cPo<8DJ|V0-KBJQgS2#BI=+|R@0 zqKX>txI7E0l}k_MHJ{iRVBzYIT89)5xiF~;ecUqs3#Xr?Wx)-!h&&(s- z{&a;3mj7kDw<+~vZr66P0#(-$AM`=M{nFx-{#~*%J;cn8qM9?F2HJoNvdV@1+8jvS z-q>Ju68`y2`6V>utDtDcMz76ft4E7&rhMstk7>ZyAB_28 zi)pRR5_QWlKH>9saB!2^33OUMcPBj{X!Q(#dU|Y^8f!AO6kf$HJ|)QiS9yS+MyFK^ z6%h$1FW!8nI~VU*CVEpdbud5)EJJ(vgN{Hz>4pCy_qPJ!ftGA+)!VCez@d$u|G$rSzSLzTBhCLa4=t! zl$0cyTWL8{=D0OzK2b~NGRmey~N}|2I7+n zqB}S`;u(JV@`X96Bq2GAa%*#Q2j(FlAb^9DsHHon&$zL<*|&N6^XE@)_HHBQ&CSgd zUJ`Wl_Olw9BPwcY#p0>M!^7p}W#?wM@GwzBAD>pr(1(YIh=_=qIyw%H#Ne;~{+6~2 z-nr(F?`6|@3&+e-)6xtL4TXh-4)-CLw{&$n_Q*dn#X! zq-yT&;r0mW#fxuX3k*a=Kp4spSg~SPYm>6EIdmN1!$C(!$HKzG#@;+UBqkteG94jK zu_$kQ@K9Es>&7Ynv{ zhSSc{ME#>J4SJU2ZO*CKU~< zXr2FM*drQ_zqzx6hmVg>OkAc+P2zRjaecNuGh6HXaF)^9+WPrFo1mbep6|W0PWq(x z7dB%84jW$2i;_7jlQ2{Q_ORe|kKMwB#!fJ$g2m|*Dagn)kcUhhBZ>Kf%{srmnvO>! z;itVH1$kwt^YAY_@~wil_97jL%}kjNbh6F&alzu_rzMStAyj#}YE5f%b8{Vw!^MWG zh@GriHI zq!383IC*SWPmk72Dw8212AE4yyX)@99Y|oD;o;#e%?FR#9%Z3v+|KyCZj2*m-nSQN zoc0F!vS~SynTdKnbBzw4KWCoTThCX!Z$yg)qLJ7n)ic{Ha!S?sBcZwMjCTzUg$H+b zb~Y*}@w!*$4o3<+P$mw9^e!#kM~e~Tu~{hhL=b_EkB^Veng4d51>ynDgZLV^{mctI zF%l}i(N6g7+INP-{cYb%pNDHxNHV{-hcH{a=ENKI=|fumZ~~5qV9^LdbEQ_BC2oj= zlaq4sR6=KiZ6smj9|`)CJ&g!0a}nB0u1#T%C1%wnmS<*6;^1C<&0)O zht=%gzkfeL2ND_Tuo4mz#Y99f#{Pt0mI_cM&}s-lC3BhS=!O(25opu57wT+%cgD#I zE3&e(61v5oeP8D^HQiona@Fehz1g8K$F;e=I7sNMtgJ*6xMQTD8Mr}C;jmRLREAdD zuZ!-Tf|`Ur#zYMnqlVB?raDL=g2iPa-ADWz7ZZ>)+9&2K+DJ|*+?=?p93M=ohohvU zid9Rg>FEK+xLH|Q=@9?IYbu9V1%+Qi;uS`X&C>i9IY7UtsHpSv^Rw*{sd6d*4!i5q zO?;X&UJ^`9$Nkw~V2&@w2ExO!exA5|g|8&c@wIm%+H>biG6h76k7*i!({&$3Mk>EI zHw_I9C+Cmg=`tND=z33Novd2S$$Aec6OA|0CH&qullIF~bOe~-0bP%DTXY#<3Ug8X z^T7pRl0cnyCkq>SyNjxuDGJ3?P^YWo)!yD-e3}<8URcdmWYQOVJ+t^2 znx$Jlv*Yw1?gd!XoRumiR$YlXBd!H43E?2>o2~sN`lgeMi!y#UirBup`}?ZO%A7iT zknt_6Qupd?;}$jSWIjGVbql!WZ(C6H^z;IRJ~JofbfYVHcz6J8W?^C3ba1~sTmnD^ z;KMzgfs#_a&Br@@kEED}j*bkXrZ&?Urt?-IoJ3{(j$j~q_pnQE0 z0(Pqc9~pC5DR=h<8Fx3gde$bX`F*&S|8g z+?}+q_!))}2fneZg4wU%g^wUgaQ^9E$Nn?cr!I!qG9OW1InT9?p!4XKE5OrpjeU5 z#9n!uEEK?|F-+NYXFRvb{oQ5etw*ptc`TromS7HfULK~gn#aIo#3G5OENTdQG#FS9 z7V3_`oCe8z3*c8>O;$o8`rXHigN4iGYC<4jLO=}5%WXjg+#XINARGWbE#OZe6Gq=tBCJxkj1w3dJBofD=Im8zRlNH5WUuFuF`(J zJ5xAFHB$0-5ZdFB?_(4Az~x}R>(3ubtkB@#U^)KlZ^3zT8LfctuK3(q10VoLMt=XU zRH%%HhX)EFkLQIIks7H`Zs6kPWSLDv>}}U~oY_}$q$3DPTg+_zOyOE!JLE&*YL58_ka7)QRY}Y9 zq7||aA2Ov9>Dft2028)dX$1kVQI5T+^w=w{+g-*rr?{loX-wsDEd#`TdWMAC$;{sV zm$Pxz9Ixpx;hw}fa=_X_-3q_M#!C!$PtOHyFEEUyT6MLw2nT>zZITP3^WXd<-Qgrc zN#907Bep6eJ=oy)x;ow+N)VMTt@Cb6#GjHqB(xl@iF2i*p^=)m`4$W^Z1ev1GLn$% zG|hE6T2e?@xRgSq#$sw03>yGg-DbC6xug1w)fQ9Kq@;#_+rNOyc6xgHR^(*Cw$1yb z3kS~-FpawXSSe^0rOWQ*4#-eIGFB=+`?DHcfutM$!SvgT{fW-9FX>MmEVuB`)0g=^ zK1^}^gs(tRz(PSr&M7s6+x3^tFJAw6sng_wO32AbPcJK}8NLF_5YlU0^tW$;WMS)` z1*0~S#zPTm0C+_cC}smhv9Ymoshog%?gOe#NVo{(^+2_b1YovcA_fHoB_^s75oC{< zot~Wm_U7;JFQTvr$SD|HfXN_7X+8Dk#HFMb+I-tYvtPY@iOZ<(S65fpq2JrzA2VEC zsH=z7wYaz#8j8H5P+~j2;{*a5S3C0t8F?4P+!>)pQ%6~OV`D?v+gshnrc5^f?BwKE zh}os6m)9Lo5g_N5Df6P)?An`nK;oLv$sG(Zv{jc8VqywIL3nE$dO@kqo%s?QtCUEu zLtCEdy6m2RLrP9TA?3>sBu*NSYi+a9_F$4yB4RcY$}WLN#n09yG(L4uuj+-vi;#-V zM>;}7L-9H7{tnsHl<37f?M)A@tO#&(&*y1^QvyYF4PtLQU%dl}BJarMzp?cxH9haO z5Q-u_CGHo0K}zX@YGhyl>9Hr`aq$8}nlG1O6G`y`=)oaDz+PEQhf8i(``aFF_J(Xi z!@}a&^MEKD9v+@TOQ?K@g;nhY5>YA<@KGLaZb`8cc<@Da)&oCd*SZ?EBx5w-6v4q+ zzElT)L!rNZ{Q}k7)$S+iC+Gk9kwe4c#6;DTc}>ma=;+UkxDZlOZ!a&4@M@P1fE}5#u zEne4^z+;)n3zy6-+C+ADcL&e`w3>o~Le2g=b5?vuusJxyNg6vjRma9o&@HHHYkOUv z8aHS!=gAO=iS;xJ6e3QHn!*IDVnub=GXcMb5+}Yu@uF6>;eM50O`RXBqRe}uMVauW#|-5 z2GAUn1q#y=okK&`ryG4_WMql~-@d(oAR{9)CrKZ*T&5dmTO~=e8~4Y7S=`7wL8X)j zXr2#jN)3$q;apWdOsU>!=dGc1`Apk!CrSd`#_sM7Y({U!tx%;{RU<>FRGtzDXvfWd z>=X;2Lr9p7f?;Qhlvtp=m+78O$O(XZ_S!AX2T4gTlbj<@nR7!=Pfy}~t`IIzE$aO4 z1Ms|n<@3L*o0F51@h|3MV`J;x;VInC764y@#S4{zR-9 zt{EMbuhAsJ!O54WL`21Bbw4%Q+5Q9t#i+Jnn5MiLvNQ<@5k-cV1EA{waiCb}==$ar zIpEB^BcR3_5B^{p;XNB1%@of4vvYKmk(6YvbGGvMz<~h-b_t;0)bYDxVkDsID(+2} z<{|--`x%z+@8|p|)c*A*5Co~z9jPRiLXMl0^O(2IYz!hr5<(ej+jd`zg;T9rnp>Is zdkcsF;M6E)0w0Fe3>~A!c10Kch1BL)wiu>@y1KfgWN7Yj5Q$IJ>1d<#9&NzR&W??Z z4V2YoU%f`tnhLOSz;&o;Y2#YspTFy|2T=nESFKz{*qrj?hugq54FYY%`qMzp?B#Y& zPAc2l+Cp}LT9cO_1tmGxA-0e8bLR`l?r5ezOvLB*fTPgS5}B zHOrW)$jU~vF*4eX zWr-v_qXkMAP#rRA>Rq7jcwF{znT=jz9wT?w?002?sfbDyvYO59w6+l0A-*t zCW74TSB>=d_s`c_H+fw<>}9foiDEU&n1aaC&3}JB@uZ0)$>q}Tx^VOe1$>@#KQJ7M z2INQ?8IOBNe3hx=$LjzZu$T;9olnRC_Ap(l1rFjV)TU2tyIl5cdwjSBH;RISlIw1s zMLC?p5gGiBR-MuA@23GaP@WQLH7ZR<(m`T?dF#ACt3aK>Zk;@o4Xma+a6;hPX*IbV z0BgbwFDd?*_wn|qO{?}crqAf{dc=2aeI1-wE{%H%aDqnO4#Y1{zUpP%%?QaKWF6tU zuN$o$^!K8giMZk2fQVs?fkCvUmRcpJqA~|6h5ec3`M)uC$7-Nb*4NfRWShNjU8wZI zMZYt>BP8T?JJwI38A;`G>7|c4Yp2Ed!(eO&Doyo$9|B6q| z#3szKza9S8d1V5}?eNe)XLp8?d?q@}MJhC6Vq!q=!gf`t^a1)fgY z(hsn!IWV@sXQX{LAe27=3_+&u;NGz@g#6(jgBdDmTrkW-2c8$#x*ETn?iHXxw zQ&Xd(|MSH1ecN-9(i({bAbGSicV|cP7{S!fpTJN6CAmnY*vh8GZuL0^a7O3iv@XRp z#3xE)VqwmQ>Z2srkBp8X%1YlPkQ0&PT4B_$;=<){KsFQ@=7Om z22HAlye$_yI`UA%F`sO8IE?7()bREC!NI|lgNu)JjMb_DSXwj%1)pI6cUgQzQg5*& z4+|?tC!5yw*S9KBS>#XPoMI7#4H`sJ9J3<=k6!;qGlBHx^St;L5THT*V`vL}*xZ`p z5&54|kB%V%h4yYbRL!LVKFw#{>;hzMgGQ$DKbd)XR$%OS$P`Ac*T19FM2{{mwv?Bb z2Ud!rq9PC-M;qpgTI2d=9v+QA_P!IC7SfayuwR4jIf+0)eFlQ!0e|kjo${OiJV#TL zC&={>vy-(h=eC*dZ~|+@T41;oVp4FwnHa@qeev;_Vk0RayHDv zJHnvXdUrM=P?xl3$cU?2s_A^PCX&*WlbhRed)Sm1ZURQR7$6t_$ zqnM{zMtBXAc0{0nTtBV`kp%zCP1LQ#YCY|z0jv;&*8QMn=7nH|Pl@C9um-s~+gs!S zq2%QU;8t1BRd#Ku)@ZhXdJ!&4>P_S9TB_YJuFt5xO9)t^004^?i=!6*55U&;lY~cn zeFaa@775K0M12y5Wc&+dyOaz`Y%B00aS6_pZz>1t^MS@0Z~T6>t+YWQ#9Lt zsnHoY{mPVB0(Zy$#*Va*8(@q->BzXaNzg$E!KApkxk(2_9T>J>--i)$jeG0`v0Km6 zf`VGQzz^&iz>0kCZyFjJ z00w06dAfii!<}XWD)0DWrD?jmNnZ>ZxR?FW%-^83)ZE+*{Q63u0qg7QfhUH8jm>th z^o!!DjpVnd>v>RvMC|srv~HQ({d5BqaoSB_iUXcoEujWI--wlagU|X;@9|~^fKW~h zu<1yAt~xwsWFUalaD!U}{WBEp8+ufmV9~Z%~L(1^W;0` zT>)Z)hrhSs!H)0}F#kW@-9G1IVn}bEM#6714`@?`Y*7u9!#c|u8xSPWi#P{Vn*Z)t z7y@t|XyNWLv2+l@qQ*NT&FYgtE})c%u*cW5Y(w zDjA|mOJzDuh;ImSad0Tok|s8cjEsP;lr5g}@bpANKoAiVTd-n&`0J0B4)`KKc);DJ zEoy*IKutyU>h){DnM`3_f$uW5Rv9JO2cAP#hyL;9Wn@T*`G)U& z*e~E<=I7?(g^T+7J_1d=%VUM1AR^KQ)EW?X+S=NUPCIMc+ot7+pd~l*gUJvW{NqVn zs3?3-l7Q_c+S68UXbilO6#C+nS8v|b65csT0SFmy&L#h{UJr&*BVxJ%FIRC6gwu08 zEAX4dA^0ks3*boLP2coCO2QS8-l2(!3E=T$5&4!A$wZj`bqt zqG#}kiycG<|2?A=u#ts1p5coYMw^()AS2=PT+})$N=SrA<|XdjAxpcDlO+>WBEg;l$;hkRA5_U~)U%qDI9UCl5#VE5ZNQ zz2pniCOEkNt_ApC*pv3-{#@d-w<0BiNXveSwD@iXyH0H%ZNhvw;N=k|0ISy z1)*9S>?>g(*O)vVI)wDB^ih(!F=dDhkES zzB1K3gH~@mf680dkFPPNiJq&H=CVypTSCOnp)WW{1J(M?w8)zjm+K$to*oeT&2P%= zKIwzS5EemuK!e?K1S|Vj9K8GB!l%h2^9+2wW|x^X>m)k9?dP1e8{}p+%M2aAQU&}} z9tYg5LSghnKbA2eABMs;%I^Az^XV2cwV5y?_b+e+{?o0Q2{)Rd7x=#4E^pN3EM}8~ zerK8@g8TcB%|1RC6POYBUFDlIMR||qig0dj?vvsg8M!<^4+{@Ref_#*od}dYWE2$3 z!oM3EPul64or6Oyzt~;P8MKnn&>|b_8s4?(y$dlDnnXQub&z_1WeW+tX_D`scVj&a zzylLsP3a=JPRB5@yN^`oq&A53IhLmBe7x z67>^6^QRR*gAxWBUbSAglS*>dfJ}BL103^R59hGSKVYLN`KdD}-7)d+9mfk=4g{Id zwM1C>$JhfC%kwlE>}<3mH0DXS=dn2Vl9Wsc8XgzWCN+2z!ind{lmT zVC*7~%le88r^}dXO-#pF*7<4*TQw?joIV_+&ChPWwV8TgvN!|Q&d_!j6ATI|yZQLX zLOamW0Yx2j816woyE}qN>lPd6H=~B^nkVbCKaR?w!3!KSW$S&p}SG)nhprImonYpW?^d(lC%C5c+0=3DQ{5s=LN**ZgR?!hPuzro{PggqZ&7FnlpGefp!7es?I3 z4?TE%Tr50sePsi)e=eWm$q6gQ$Hylv*f2oLacCx%lvgFvnuP z*Ek!1bvMcwq*=xIc^rBSYA?jxBRoGG->>izXONzqv)~xTm+7UR9p;QXe;*WSsJn#1 z!Tnus+`Q7eM@5%}Cw6Z0iN$^2q4AAP3b^^L$7g3%4HCJE4ULVOGle_~Sc!C6qEKLy zC{bD*ig^HIXaP7-pzg}1al3nXL=bQsvJ;Cl*6>}8#f~*kOOv|%cuaJ^XZxW$9I?TI zg8=-tgG=Uo)DetQ>0)cO%eJ#0?ZLrNQ7YqF21nh^wbRjpc30n%B(@QzyjbrAjrqAH)9ve2B9PeAar8II5XKXimc%>@PREKg7X&}I}D)K3=t8)eJ zZm#UoJus%g;sLTOK(|&(WcBs+)zxtgY01cPfByUnbn=t`3>FB@18ZxZVHlS*%Mx!V zlJKFr?#!<(J}uCiMS9Z8z5RzI^i|Q}Ae9RgSML_NL-kNw%=XX-L#o#pEG+SPbu`kl z+!kNb+E*P}aYGr+M;qa_PT);oh1xpXu1ZGG@)sC?2$&FZz<4QI;kKME0c?_&oV;7H5-5K> z784^=Q-PB|m{e3$z&NYX*53Ws2yDc*WjBA$i0%h9-1D{a;mENgmVf`KJ+@y4D2Pqu zrlxA`UyZH|>=;E)6rpS$xLb!~yXZMnC|0{*(=+H!}Oy4+0aVf0)@=E-$KlqHv zzK+$VV7W1j_}`E-L1)A5*hb+(s8}lYif+MU7^mGB>yCLP@U=E=Dw(p>m5ma7LGxmt7yx zWjOVz;w2$P1qF=w;ZrOLlddX>Euy8McL`QwgxK)6cmv0hoeZqQSgtl#0{j*jd};ut z2U$iY9q#}AGlPH{QJnlkufEObXeIr=A>Vh0YpYOLH%)o)*cYu%wci$_h@y9Hn`(c4g0kj=tjH50yWyvoTm1*#qve!DtPQMADCd!V7q|w?Z z8aj#11c_2FT$te#wO;ZmU~M`yw6*yPbG%5VOOrKaPAWP=cnOotWW;0w>u)}-B(H>) zs}*FV;g{*Hl*`Vj;f3`6vDaQ2QT-)P9ZwxEc@tMo5f)Y&GT0E zFNOYk4lD+Z#JOK3pRq{IFbuo9I_xxxY^^mfv=IY#>7DZmsRYy~NzC-On3&c!-1ciP z48zvzf$W~X{|d|`KIdJmrLhY&#a>6p;?M16^txs}hZPTM2Tdr=Ww=~ts-o9H8o*t`cM|kvU zR`{_`>q8G404Ng@5(d7;6Z)@*D?tt1DkX7o@g)YzgJs}L73;QmEU8KXkhwTql7fPv zX)Wc^Bp~VY9Tg!jJ{!IMsjo|w{rkG)xLZd^MAUbi2@;C>H@Kh(b1z~+1p23MqbS1L zdwclObJKErInv%iq^p4xARCSA1FcLaqRF3b7vRVV!Iwdrm-aY9eEkebN>XCpyjTRz z?AA~UT>wy5QVFylvT*dg#u`sSc!0Ax1^i~vSKmz>--9XAdID$N#szu~+MQteosrUy z#6%90LEKW+VrA;5MR%@p|JXSP3Vq*GFwQ5q3sX$*7^qt0h^x{6Ng~afCMLwtp{*IA zS^B^_Z?QOk2L=$4J@4k9aYTeF|!h%LIAwk)z!-MiE>HInC z%Kv$wCGhD%$Llkt9>=gv-^5c`G3s*V&t2x5Jc=4G9G0PlJA@t1Qa*r{zJDJs);_Fm zORcStZnW&yo{qJG#f*?FuxtRhR9?W3Tk*b3%e4cG*Fcb?!QzUx2v&YhU@d5>S^=Me zn~O`Q89YAck2v!X;| zJutA=yN*OMV&FkadQrpvP#g?-=wHf2e&MvnZu>W|`hZtJ798YIu?Aq5JoLy~j&8sw z`1}8UTER5p`vBaVG~j6fq`bd@RV}}qGyt>|tdF69=;;As*xl2^(rvMbwQK-*8}Km% zz*M!cumEe>#>~!Pumj-X0w2V^^%T86|f1eI2g9KOX{-J=dj;~tlK z&DC*E1tVEyz}_NkL@q>fsc^1sEG}NP@`eVM2kPjW_HAUe`y8(d8oZ0GenZkkc<0NNg>SBxINTv zlhD{n=6AJ;L+zz)3dX&C0EU7G*IvznvqA<%UW;R_{qG;LEEnl<2c_|XVCHi?vtaU!Oy7pEiR;QJgr~b&HlKY6&%1En&o~AwOh=4 zT-mouf}0(uEruU|?fOXYK=gjdu{_dT-Lfng%1ZwAwk&dg+lM(f?42HZ8xMoxl*eyB zS6_P)exwak(jeC`0B)K9EI&8^`<vV1c*kH^k zkZoT&j(Z5V;Ru`on98@YCuC^)yf~`-yA01xi@85(S{g@ku2uWpP1BWHSyraj3pl$r z;dYBt#GvChyhMYB1T0XBNEgWzBi&Cg3kBxrWa6Yo($Uzp^!uD5mbS`-?99j~zCQrh6)?KNs3`dSW3 zGXdQjP00L=KLN8fwozRVq?->;9?xU5aeJQ_P{Jx@UwbHRz&9LC&A|j2Pu6G*3>K$g zt?S|h68Oz?RY%6B+GaVAPp*#<-2{)o9X+hAA?VN=r*TuApxt~R_Yf;WTN;MIt&l-L zF8zevv~Pr z<_+1u#}(&OTq57Ir_`DLTABFc9@1I;;$kB$8n)V>q0gu^E4H5RT-qBz7@mcvTS z4L+g>{au&hulhrKO=fIC`*xSSI`ry#Zj|F~nx=b-R85oG_JgOW!ksUEU;XMnywEZ7 zM1W#KX7q(>%c^Rucn}IM22l~{Fwqp=L8JztqK~}Pi&n|WMjD}R%~4ajJ;`}RmeQeq zjZIoIoVR|*$L|~uC~`m!F-x#Ze_D&!}&#{dhl#WOrVp$s*r6 zsMpirtc%iVH%v0SWJRS>8d;En=?UE^kv}I{wDp!)dHk4m{nh%-5ND3ue7W5G5&Cq0 zrif)y3eM3Mnd3=RgXdWpF*6QYqZ$R4%FF8AO0%~+cKop(6=fDbdV9yexEB4`8G=H4 zN>$|(@ON&%uN6YxsW^ZAln_B^WYu9lrl-QCS1PJi76OfPh=q?hHyKCSFm@i#?sgK! z9baBTUsXW`3Q2RiG;diR&di6E+yfTONFU?N`O+8W;_o*l-Td&Awl^f@tfoCutjv}vcN5-^VxYQYZQJjZu`NnFRc;2Kr*$TWI8M^Ir zk0-Hs-rQL^|=CqH7)eL%8RvEUTO149jrIm z?(5=4DBcZ0_glOv%%LN`uT2C?$==5Vf*OoI6TxdiPsu+Cs`^?R8^>B=`mo8a}?wGGhGP z%SysJ=CQp6C1UHqtE($FkUps;W#zsDsrPAf(ghv_QVjVZJmqwx{)v+%gV14zU%pKhtyE1-lr9m_!E$W8@ZSn36;UzkUu|F&G|; zj!nglUX})B)Nx+<0oe*Z0kBO={MvtC{}uhr%+!|Rh}fo78dxOctkXJJl{eCdDu)_| z!zKUZ`K~d(ko?}jz>n{P{t`~74rhF7WlhNi$#7P9YuCk4_dPubt0ol&Rkt7VNQBJUbxHgmT9oD|gB_^2#z9#P2J1N}&v ztl*5GguIE4fw8@j(B;XfHu6v3HhbJKa=6&>hpC$%Qn?d((!O2D?=O&~@s3}m84hCH zVjJR6)9>YN&vGr*X;mK3b7CRQgoU%8LIP-oe_w1>z+oQ{Qy*0fTa_hC#{()|R4yR> z_Vx37uaK6R>%fd0nC3Slwp&Jtb0x(UIN9CT6Dr0vQ;A)`0|Yq>DNZ#s{V>!K!=2SH zGr5a)=G)kBQnO|Fz~~BFy+AEKC|{;kOo|RJm<+Ddg!AJG@^jYv%N6RfE{juT#KxoB z4i)tmb{n{glay7`I@L?VOw>ki+jE?F_#8d9!MCN&hx&43dC%*+%|_7khM>bWvvdAu zck?uB9lY-je13}zV^u}QyofMc`gAS)g)rLIsAA-5X-kLvrZP-x)_-I|>5lBA0MqfW zxn%u`q-u0t^`8;#HB;)t)g0n|)l*l^ki=j0@gjSc76juo0og&B(=O<-lfq%GUO-?MvZt^h{W27btdKuO?E;(J*lcdSwLFp12TE zOZj7zR9O7u^KSGHaP3Q<+8gz%9kxMKHxz66?*mT+`}W!I)VBYuD0V%%Bu;TM1qVNP zD=x+l-Km@}z+DfS=*>xy_}Uy`bN}ec>(ORl{myEbp^s&5qqQ8f3@pbtkx_DT=n*)T zOguD+BM3Z>zJK~WuBn1kzoUM3_fY*HkHjF`Jr?nOEe`%`YCc7FeK=02aN*#L@`ec| zj4l4fOETT9adro1&+X7S#j3)r&NL35SV6`ltgi}vVGbrPIIPeO84E}jg!!FHt#!e9 zo8qoGv}-c701abPX-B~$^dHrLW4#dc{3}eU<&(xMhhc?2DnhOX_7|FDqgF#gkS*UTB&(wVYzbq@m*lhNzj>wAi_X$j=Gy@0prA1;>xumPYK~~ zdg<6W=@^%<<3$>#E@*J@yzN!@aeAF$9j`Uu;2gAXbsDd`m7QYYoruu zp)$#ws2$+?@V4#!7R=$7T5TtX++qB|-<0fQ@@0+Wwwk}y1uPRQ?zBn+xMQt^_C7I` zjQH=mX^M1lx)2kJ^R1KC*ywmv+rOq#yTWNilfP4q8v1(_{rsXIV>R=W&5?X_E*u;> zKxl!SfNv*>&{%PJf37`|hZq1(_YFb^DO#-yAtbn742_haaWL#4NpBp0{joIumM3yZ z3d=3RC%7~-Qf7D}h3J}tXWG1fMr_Fl&CR~*0>%5Tp~2cMJv*x~UQK&mj-5`hcaj-$ zQynxY2MiU*$>r~)S!QTm8O(SXeCSlSOnTyfIfLXIdjF+l=RJ4T9F-$p?I0TDkXk70 zsdvl>i^GgY8pDN%Vl_<~VTEk>GDlBJ$a1rAF;3{~tp`4AEhTji_09izj%hktwm;xv zto-rak11o#Sn;x~;e}|}3G3kqD|PQat*V{06HW~zy7=dIQh@=YZ58*Z@;`dBMTGDj zAyLXP0UNJ1Zr!BZ;~|x+9G(}tF0UmgUYx!8kc8YOqvmeDVSgyN`fId*&2~V zkw#f@cMmwr5$pJM!sI>+T*O~f+v?2eqE{T^jPsh6CI$aHImYKd>lQnm_`j(A6SA zU0ovVHKDi}qr1ZR>{Xc@pHz^TaR9He6?2KQQD0PA5rJLt6?C~abF`2U90jzEpp)0# z9n2Awy-89GrVvs+GHg|J(i>MCjfe=k#-Fo_1-(&ydd-yDl*S(^8sHI}6nW>8(m#5* zeb<;*Yx#~gQ%NV_oAC*{m(HT@BsQO z{&_pPdAm{uot37kKYc6=NEh-i^juyxXupk%BHBMd*Z$~A0f~;D7l+0bQ$V=Y`oE@f z6>by-UQ;&s5@XNgf5!l;h@zvXeDa}AmBDn4?}-P&uSJ_N%VM!Flo=Ept@d3e@>1Tl6uIvjxI)qZ^ZbQ~oQ1^Cvyl1Cu zUX4^~lB7e2m3T&ySyHhwIkQ0Hzz=itA7xFLbqo31=-hQvJI2f3B9JWE*wcHEdj2UdGQeG*R=Y9 z&B2?)XR=N*Qi8InUtML#ICJIy5hw1=sD%WI8>;4f;OYE5>IB-;2A}y`YVtoK2$FN{jus(~D5R31MfCaf}(eQ>xXG6ir&^GTz`a z&Pk;(Z!5!C^$KPZ^FA~xVK4M`H$d{1MDp%OK`54w`GgOO!9U}Avp z8x*jJLAhBpBG9Od3Ys6-P@6O&-^2OUu&MpodDWTWznP{|SGITs5MJz|obHI$bi&3V zRn{;&W}>?z{BRU;;N-hv!F+mGu-Rv{Bv>mp0b-wY*5EJAg{Q><9Re7iwdl8ZHq5T` zUMo4W$6F`QSyf z!?_`MKj&dqV+xg9U^f=E4ix{Z5tQ3;^IAXzaX=qZbJpA?hYjZ!9D&VPnk0)=Ih=X< z4`a{yt?WP+ERKtyl} z+Q_fnjx{W=&AL{}Q@fu6NPRPJ;y|~B&^Vr_-;>*6_>=&DediMx-N$kw0sfdoA6T?1 z(U+llpqTZ!1sT=1`H0idYs^?IUHRickHwpJ7-I$RybM0X3XQkxM+B2*<@9WNI?OKmf6a&I(z=&%gz`(4 z1<-g;Nr^(?{jXO)Tba<o2=>|665dv~M3bGD=umU#iyE>Rs&m;cxqOWx zuM*K{gZn2QPa{3OTjZ2q64`MT7#q2!14~~>A+}=3PFW5Uu)@7uQ5P90s4~R$w%vCA zTOTRM!|B4+8d2=`M~+6P5y_Dbz3T|QgtfWl@1C5&E3rCwC9i89{I3hWJAVne7=6g0r zNzYcw#x{x8yt9dUQ^2p;E)Bs@5VNH(4Cbqt5Pt1(h;*b-zQsVId2uEs8Iim3>H zn^t!4Xpbe$a!LP>$5(uPx-@7s z(n52HQTm=dEh5R2Eg6J>pnmWS*bfXIef$R>_HcSJqD8k@bGvREmP^0NAQVqLVjJ`e z=txuT?%7Ttc;vAt1qLciUA<@9gXe~`$ixW_rp8}sV}y!&bEF_yF|9OGFM|JM>yEF=1hp;vFL%htB+HJvN9D9s%y+%@$11IT0%&}L;9ub%3jrgiQ^|3QJEr}>)VWw z!J=^@#FK8b-+`~;%-WK&=CRvo;gC#t@_?6*OJzvae;RByp<=H`UmW#3K^!`>(#nK{ zf=>`V7VQfr-1BH8EBRLa#+5uL3&8b34O)% z&MWcGMMGh%fT*)x$D}2xpa@AyXUabylF7^}LHDhv4Xf`0rK#AG05Wy&WSlQNaO>?n zl^rAr$~iB;c?m}DnKCx2q~J5Cam(49JWx$NR=F;pC@AZY2(T=0@O}97kIwJCNEG@1 zVCt-cs`|daf9R5wP`ag&25FE^>F(|ZX`~dSOX)^JKtj4f2`TCB?hc7(>*xEMdG23# z?#!Kg?m2s}z4u!0_iMF$z=gs4CKQt!LsFR{wIR`Sm`CaH=9>zYzKG4XU}CHYs*Lne z-z7K4)%_Z6!pvv;8M|4Rwt#gGpRJ)5Pm32EZGv0#u4_%x5iI_9F1k_$9Wc}HdMGMg zjY{0naR?}K3f=&{9bIj#ZuLwryU5RpXeo@^)mp#z?UY!^7qnt1Hd~3Ozh$DaWk&|Y z$o4Nisv0eMbXOY79wlcOzSd4M5b0jcCDtROz`uKs8E=h*H1)CfS39%r=P9K+p#lkb z{2&ozXhrs9UOfFgM+e^{;L1FOQ_y=FEnwJtth}_-$DV20mh4lv#PsiJ!}roIwC|4c z)$+2&;O_Y8m-MRR>_vXcjZV%_UVS2mjT9k=hY0Y?X5($Nr|#GoF9&Fxt+yn>g(05l+GQ+mxk*Z{Xty5W@8~CYwA}^42#zHPR0sSIIfOH zW`YgEf;w?yJ?id8h1_Ejk!9308kk)C>#o?J>{OBH@18T^KH4825xbyT;C<{Pq6a znx*7GEv&TxY1HW7gNG55!=R_ z<40#gW(~_;={Tv;d&kk%YPGYl1x{EZ|DMmYgjdR55+tebJP1oHwW7YDs*AjzFSdwg zKU{P;N*bVi@oeaa1#xF+N=a-+J(?~ey6;*%0e8^NPgLmv+L;FmbugGq8a#ZJn*pu$nj*F6X@qIWbn|DNy%4Gd_+KpdvM z)1Q;u%6dIJvEfTIg^NmSuDz9dIn9lh2{sahU_mf}9MUfK#4dz3@ZUp~wn4y$+OK(S z#E=M~8kdMVBNWwm_Qbon=_V-q`e-6c&j@B0pN^W702zzZ7vES;L1Y9re4O(-mp*uZ zb-KYq(M0~=pQqLrr-!x))Ypi+jgP>^TS0*`Bwv2sz7q79lvJp-GR39Qd_klHhpp8i z7TZ#Sa%lDKyGU5@^~JBD=1t`~EFC;> zYw6F{eZ71>qC(^vaI&eeBKqRRR!OQ4BK!r?4bF>$MXdojZiB5O#vbw2LCX9I;M#$MQP66He5Jgmrsl60 z%iR2g1{j)^q@{t0Od3KG0apNfPq{}up2pMnqqfMxt&j&KceCTf>#1ROl7z4a3_YdiOZNn_f{eH&Hlo$DPfT#B7nmGX%DZ1@c_Zb9|oK42MXRNTX?!H8!p2N;cWEU3_q_I~f zdwvbvKYAa1QV&{!iYptY%`dbKYZv8q$<)hPG;Jx2U`qj^rm_+X0$NqZ*`(OmKptR9 zPXS1mlPI1=Uf?+aP9+#4Ami02d42aK zfnweAqB>6H_UlskDjdrKpJkQvkSwDiMJR`1nD3US*glOVqAzM#`$8zz4zKa{E#Kny zSW!&nywOW)T3T2U;1FxMT8u;k=qLlP6$EHx?}C8U;K}$i5DFqCRu(0+9737kbnyI| z6!E_3vbS*Lw{HQvG%ld=3Xh(K#)Wu@~J2-{}uxFO(s zKvbA==W$t=ZjOtK!~Vhs9v!~i)k{>!J4~F23ld4kx3ZK=ep7hQXTc3IvFIO>vt~(V z8xYh=_`&_|xE5?4-}6~DP|du{(yZ_lcWZ}FUU2wHgcvNU<-e^w$Y82kD36^Cj3gM2 z+ZVSYQ9&Z)HChJ5B1KEgZPi2zf*)^9-`S1f-Xvm}7_RhlS6@_(ocze|%^kORPEDcA z*%U$lGN-_D#w4=dHz4PVv%~@i_8GFj%nMjX5n#AmMJN8yQ|)u*0LAv10UZ!_^4U8k zYHI5LJrLC0WLn>B0>ND7TIZ6xw%~M9QN&m4b=x}-mbaZBAIgv&aq5q+VX2j@DR+s{ zi+n%tliwV9-VMB%*iEJ@R+)icgNIESe&W0;HPX-Rs!(H5bLggF;ZglDBG7tbMFCn~#)7V~e|a_# z&S>?rlUv;r1dTMq%NqeFv=Pw+AecPf?y-g_tE*3RhN5AMK`#Mc;mtje;;f*9s2{SI zY@P9?t%$E6x-2pD#cRb}tl#O6MGca8^Ij8Af@DUVGGr3ecjgUerkbQal2gWhkUZ%Y z3GZS9mPz2}nV!}X2v9Da5CW#h+wl&Z|9kXU_gZ$^`f_k9TScRCnL`G&hAzzW7z5nXKt^?Sn ztmC(E-S8S7WZv&H&wcWJ`c{zZRj!_A$_Kg+NL_f4id z+T7pX&mY5(e?zt(wnhA7LJn33fH^Yh3PZ13^#1Ge1rjb^V8{jGQTTG&G}nevv%EV2 z9l*_=P5+G$lMBM^kGT~eb!6!Dcucf^7LMf^0;NPkzP~wg$w?A}b zZO_-BDCE1V3o_(&T?CfV&A~IhEnvAZtTcj&r4NDcbXSr-5T{dvQ9DTRGhGj6S-FJO8lW6rBn3hEh=`6Rg@@;TMVsXpOB@Yu`hGs;_SNR|=~04Di0zX- z7RtU%W5FI#x`p?c0M5-CAY{?~gGd0I{fB^95pqc6{kxLyRCwrd4E<`QInmZg2CCnW zboA{8c`FV%;M<%&Ae63LUz^x1|Ed1wswT_$=~?AVWrvG4Q#3eCnn}Z%1~Qe((|-EP zNV)kH?Nhdco=CQ91!aR45!ABo?rtvINvIuo965mZd8PcXF*Zx+9vC8gWx)D^gM$MM z`1t=8w@KvhM#US)V_GxzO*YB&qPVpbE~jz@0jWw4d50V!63Eg|G)zt0M69|B-8AQn zXZS8xyu3wf9*^`f`^&!Bb6^Cx%5nP4-qIHKK7o|2Bq6ZUiKQ;f9`;MIW?G%2zz4Zu zm+RE-0*m}~y|XDGv4w7El)3|~otQfU1f+Su%}vL^fVvEUHNY1BXG5vgHra++tB?&t zuJu4@Ub@UMHd?abOJZNQAbx{g8rB=lV!dw1X#E>^u2{1!#(*uCEE-ov6<%cg!4-#V z;R0vFqKG4j*3%!vEL0+-HIx1@x=cQ(^=exavApp7r?HDGhsXiQpW4Nc0qg+@0{nvl zjndF2V6CtF^-CTC8e=8Fi*azdUO>?ZC-U~~4S;XJaY3*_BeE-|m7Fd`WZe=GpVo|d zD3Wjd4}|WPO?%ObGSgY4Dl#lm(u@CmJ3PfH+`|fo24LGkMOPuQ!=~`BIhmllRan}}{(*sSe6pah z1c7B>ISQCd4$>T3-vYw`CdsRij9}oOuGOL*$N;eFo` zj0BSn1{d)5{0t?GLaZnSHr!*Ik-rgAg0Tn^Va|h=2XxKk!p8+uJi?>C#7dpl>Ju6D zg-AwrtYQ(fNItaS>hh|LE}u#;>-QAAwGS0SY~Z;kAAF)$?9NO9|J=KqqIcYf49T#o zGeScpYHG_-0AgkQ*2E-t!2u}{>?@Vk7BI(vLvjs$ zUyvFLd5XCi0d0&w7Zi@N*NLcJz_$CT`(%$YX$5RJt$pVh7{abogsdnG^gQ#dmbdZA zEFGnd2u-ll?|9Bu5KKO_uz(TM$$$zS6Y>g#W0L(K`u{cqZVwhD6j;cjqVIvYNuROO zJLC{r&m&X8 zCX3x#`KqMQ3Y7+dJjdD_mh?1Fkaq9s5r8O)24cgzPRh+zkD7$QWdK4uum4Nu%^N-P{3O9uE9vR)2x z>IckAp-uX1H4N`Pt^O5vl>y(z+E>7EzIfxUH%eCFgfcV#+2ye^>c99G8UtZDzj4eQ+LceSVs`;%-r@zGgPONVg_j9+im-U8x-vS2ptM zfx%5{n13>Kw7{CJAHq5~W`s66qonBkgc{?b`(RilN( zEEG+Jnzn*c2X)i^Y~*pw{r!!tkr;S+ehBV9L+aVW?7@R_&k`Sn(7_Ifk(wHQm~6h% zxC>agc%uQ}Pq*HQBJ&g5zl}w zqw<~t55JBgjsJ}|$x@`~Yx1X2t-$jPwZHsz zf3-x0j}N?UO+b$hBO)!`3qD~jX?ERGgpkqD_JDgG`Uy@<@8DJ%LnJ?YC|x@dj@T`9 z5a*?MV*0yI>>=QtJ90dlRPN&4P=f<=;vxnuVgxfXG->V(k|mqAs~SY4&o{5E?tg4n z7oLrNxzuLAI;OZR;w+y>l4H)dp!e4i4<8r}_DQGG{`Z9R-(7 z&ID_oRBNy2U~2qyKBc~V>nU!-OJ=w()H?l}Br%BV#SyDrX278cDiib<^ugn}C;(hM zz~kk)H)#*RHGo^BTWJ*ZWfqV%jW`Kmj8?lp1d%`>eT0EKgjocaBVMw;a?dogdrCZv z6k+1+yVB@_?H$u$GrmZqEP52^t4bAJ-nuvEg=K~;YQuHv`?T>7Wk|I)$Ay)tVqnk6 zUXgZA~Ey!{ryr0 z?*;trK9UC1ccSFAz16!NN0r}R30CBfO|yU0N+PYiI{xf3E*x~jUXX)6tMwAz&0PTO zttx}+*9b^W2dKq%=LY8IWad*#4HWMpaSy}bf7sSeDlQ`=vjWvW;39(;eNw;0b8vgC zR_LfPixlFXLfNJkt4f;Fh8*=7Z_}ef!Kft2uyi@GfLn^v>e#0|w|~eEcde*;E3;2+ zZ%8@b&3w!?tX@w#Ueq_QE5ic2B3Rb5R|kVrq#<>YLLyz&5q$K8Qm*~57~jgE_8Zq# z7#3NI1p_AqME5dsKzC7_`~83WtL5hf>nm=OuHumUFy)zhE)i%Dho_#y4eA(;zpv8}*M2zwa z#Ra+QiJ~}LV@mMzS8fd`erBSGuv)H31Ge%3&LUQ?gL<6xWM%t-ImRED+5QqPf%6V{ z0!%*{=^g$}5)u|8puHL2e18JHBxl8jh>53p;UVv)G1ufrWka-Nva*(b!GL%6`FJ>K zXgS8TKS#1XKPBADE5FlI_gY2%vrX|`%VYve#VkyT#f3#dfd%Py1)+RTCw%0PM#Yp? zs(tcm1Issc-;L31%0O^uy8QVYR|G@^7$nrud!*?UCBz5RL)O=&a9j0uiO+ojJP2&n z44XXc)swDyfODo7jI&IDHUSHN16w047IH_B8W?>)dQl?#Co!1>%032s6lG`n_&So) z$7#M^9CtmlTwJh|t^re6c!_^lzeFtNs=)Dme)E)rP(FTHz-inEQBRLEmw5m1s+*jF z{h>vg6pv!xr`{X5{Dm!%r3WlLjiWG08o1?7ui)!43dM&>JTu}-4Orfgh7YSO#L#_7 zb||+NlYau+C+y$iWO?z6I~xq1tE(p;59p8a%K(7#2|zmlbX`&jBlPmKJTTXg0+$D% zIdxWHNIu9?cNl}LGzMmId!f$gVu0H`vHljQ~fEH*!rw| z=PUFEJs6FM9owK@R&919R# z^1v(D{T}TDB>@BG=OX5z1~~A0uaus=2Kp>MwJetl;_+63y3aZyF@gUf-zE ziKbQYqQ-JACr0jETBiK<2Z>CKOa73d;7?Bn-`sD*65OP%>4jEvG{P7@T-1IJ-tO1Un-HThmxdUJ^OF%*bC&Q6{2t6=tXv@#m z$Z6*o|4$1LBTNfcJtqhrmcq)aG^pBjV;B=lI4{0Qewog@>O`K`wp%rMz5!WKI9OQD zqD^MQ3BE?vOWDr?SLc(r+GYQDy~D-(vhdh!1fQK!wj@RhexseGn9hyoLTxqs0#!I8K$D8q#qn5vi1 zNnWYtjeb!p0oJJ&0Da<4t4MhS)+f~kC*VP95Xi5pIt5?apZuDp$AXED4p`NzzyL<% zg%1zgL4*1suTl>?gH%wYXU1|nW-XSWy83ya6amFzWYz_7$q9?7U{W|=wlT(%!RDm- zTb=OoPQ)+V*yhPF3)ckI3>o~ILEw?|&G*U>H1zIqP`4R|>Xti`tCW*UZRAMhbx9nA z>5>oe&cP-yjz$YO=Up7U=43loW=pLK#xCxf9JsJ^KZ^-OXwYR!{=0)~e~(QCT=Sx1 znfY_T9Ho>g&;V>{0MY_@K)^6}Getj81&-cbw}pV+Xj6L&kVlTqCqDrD>gr&Q00(Ea zHWWDOT70e==O>C)UsxLhNIFP{fzJRy8l3jrUg?|seXRyO{#m+O3Rr5Z206aN*s#XO z&waJ_{NEgVLiyc3Sk_xFcfQ;3*iClj3Y@*FQH!`aWpQ@m8r2N)K3VR?!E8GU#W}X) zlPwl&ItBZ!8s-!tY&nJ9W$qD!*?hUWRO#Ar#!MNv`WO~s0=TmQdK5@=k8kDM%2MhS zzEf(LhFOORcC$}(+Coxi0ykEB5GA$ySe=&t@VdXu)}%sw-J7YkFMa%yQpV?BCrw#@b|Y1JAeiDHyZ&w-GJnYM?e6&ArSL>`~|-Kwdfr- zZ23AC0St%_b(uknjv6|J8{qN*D*giK0RW=wP=7xIGjos4X)vbHbd`DJ(hD(iEImp* zEe-td;Y%j}_IY!xNn-ZA*qD!Dm*zP*EBzyfznBk?oe>O|jH&g>7)A9bU(UIL15`f; zMbXBc&EqT|Jj6|g*p)^80;OEc;X4tEVdF9JXsEN3Q&o+H1fhRP5a)?{!-Rx#0~z;A zTsIA34!B2IGafV7yPq_cGoHf}zUx@>3mw-PKXP}(%ccxAE(vlu4%cw?AG~J_-@|6I zkUf1Fq;6mk&EtJ_cp&H3s-vy_IY5x`Nf^i;BSVXTH2^R>tDS&$17Jir&|eS|;D3>& z0erG5^+ytZ4YdP6r~<(3dOa49YkwaTg9Lq+|86y!D@Epq3jw_O>e2@+WO&aIdO)^l z%O2nz%sgJ<%swdycS4B{ePs}_D9|-4Fm$%%Y}s+TAkj;P_k}F}eb`&SkQVc1w zK8=Ti>ibJYv5^}s*3DO23&X^wZ{4@wmN)Sa=UM7W{zMa>&!@`qL^IC$RnC86>gE_@MZc-!D3#w{AZ5W`oI zgo)@mA5LzEn<}g|{V+bY|F+`v>M}dEzC|~T60U6BpKnkt8ndpQeZ1Qe?mPuOEG~B^ zBF$$26YzSfiXUvI(V)5o4_&}=4B7CK=mB)m#Kgq*OKEbwt$AGWuUSk!j-SRlLo5)v zB@{upFU4HONf+43puTK5n~z&fwftgfuWVZ^wtRQ!NpF~nDEX2a>q)SvvO_3Nq%Gfe zWHfHlaVgd|Xz`vW?A-9Gd zwjm1aR^O8Hn8|o6Z8c;~(O*pA1j~PTA=3VJC==yVJD~r;E&e4Mv0h-33cTle<*~nQ z7*ih#&bQ%VVWk%vXvZHme<&#}YHvoKVznCie(c1rg@RN~F+L!mb^U=aD3{+c(pw#xPtu9v zRU=4cw(pj5&7SjLAl~q?ow>+}+O|P5^r(R+VJi91ikbb20c}1t7~Zo+l8@+R&;&p% z0p2leJ772+Oz;#1&^Cz8ZUijdc)mH!&3B;wK??pKz}bO9 zREtRuYKwuu!lcnl(B+|i?;8M{K!DzEo)xY{j^&FmOtX)t=#EkV1+KJ<_VGx9hvZXo zDQ1so&!p@X6QA;osl)I^TI?{J7EHjZ+|%Sn&XNzzT0)pVGThFT2lcC8Azk#)D4OZw z6n1)A^TA!Iuqtd7NTrPL5lJ&hg-2qy*(pM8t&cQ@!rbIyg2-m6VIUjy40?jt1;~{c zWJ3GpI-rNedS3z{l|x1kcQ-m;C)*AqA|qkQX=x=J(Fj&iTb04|1o0_7-FKxk6yN{_ zLHijHqJ`YHV=EP{t?4n?+-FSP+&CdH^saxCbiGt}Saz+!L53%7woyn#y?s57S00%T z3wdh1D2XGHW861cOW?jf{q4F%{2BsBrPfpCtekMo+98Y>GUK-u1(x})I1;y*j=!dR z1YD$vKZT_C{D`mKKb3R;fRk=3TW08h6G16jEhRTHY3r`Jc4c?t`eNFVwW(DI6i8Wj zHITD{ONLms-|V0cg!*Y~(Q!v&c4jT{b{(>=zYc}zaLw0YtZCMp!cA2!XM1U=10;(= zk9TK)B?@?s{lmkcM+)fVv369y77YmhfD7GgR#j3`0{$GxVc&r>D@HgR1n99~b=Gfd z>UyIGFpj_VwctPtp71V;{vH~!0+j`j-~x1BOE51@Oicl#0Dy;2+8*yK^7G9A%nUHD zcjxL@%K-@XBrKd19g$~qU#D`DAK;qt%~U{%OEF7m0dQc455VHK%&S;2RXTC;xp02< zo30teu&ni54B|S`vfeEx;5-RtdOmjkVo}glF6eW~_uo)rpB;=cY@lM%)bxT79iJkc z$j&(;NhA|DLSUzZ=eC<P<)h58*9@cYhVT<)5VK9RG>SN2tY8AowZn=VK5{pah&a^h9C)BQ>M@(6qNC1} z3{+uADJbA~6pQU$OCv#AO!yhv2-zc`GN!+9#~3LQUE#a%eCFU)hQ4DC#7Qg$0Am)a z5FKyy*p3@yRL3Y$K@r{+o#NCcbnw9gXbG=wn zQjY~fS}^lB)!)el%g(Qe86JKlU`%j;XGt-e(kfu|O-oAyLxc$2gZ_)bA62`ls4fI} zHiz(qo2biYCCp~>w2XlfWR;4R-JWRB_J#LXIFvb0Ufk<$bj0A$0%-j1&GnyJov*Jl z;HIRIS6bl-@;6OZZ9FiX4DSw%Yjw~O;ea57JmjM!RbXnH<#pUL=~t#NW?Td*#%d6I zLSO#ax7*nOC5r1X_&$P?sXb===dC^$7}Gw3x6%Q^d0nCKDt}8-`ylNN1}8WNBU7Z4 z_v>$f_6uO+>~T0hb96`-@hJ3zM4P*Vfk z(2$tKt+dmTys10f>)LRR#Ky7_;0*_rr{UaKkz?qmGioShuS1S94FY7tSCGtqFI5ql z%HA9AtTeG0U>RdtKg;t>|7@oKp7F$&hT18V>2O{6Kw+6iAg*zXQew28U_Kb+2yPu`a|i{-SVpB|iPinF8)*b;SLJzETQ#Cv1dd3lqdLI+K>zRAF>;Op*I zmegReL8_AJu9O9jfffwYZ#Gg!+z8_Ljaqu$0#K9 z0BVh(r-!_>v=ja7mkG2Ae_4t5L#z=UgauVqDvh8D4hsRm?Q!8r2G~}&+?@_ov|P@D zBHO-4lS=V=)7d;#+jK5Jh*B^{fGjMeh(M0Di}ur*FzScnu+M;*+Y2q1r~Jim_ojp| z-0tn8iQDq8w?==Q9eg#v>DRi`h@(%0n4m9QgkL8wBGF--dSM`jTFKG^D?YULQTES( zAVJ3m;nO|$Pb$rwyp%f*&K)8ozw!{(R3ISt|ItVB(bZ zrEh)nW*I%9Pr6mA2E#_Zd8!1oJyq(~_N=GXghf22TxSCOX2FC_k}Q0?GVa$2n-eQ* z!cr=DXa;0_08hY$4|aJSRU(I+&+l^*!_#i2UOd#V-+%N{B)~y*xZv%fdVY6`VxT2+POZQ13=rZ;H>LmQR^{;RJ(qYTT2QVm0t3IFXXsw1bJf{Y~5XZ!@QF>}>cdvIdJit{~;Z?@jw zt3yRVYo`a7tP#3*w!{@9uW?qeX5#{ah0fPxVLhwhqC^`3=~j~YDiC(e@m?V%loXw0 z&?m4#(giBkwTWLZmsF8SsihqR0JO8&=Ux><1-5yb0|Lm%a@{%>LBV?3a0i*;5$kMF)G@pf9!$IWcZ>06@8&wz`OT?(|N^2SdndW=tU=rH|_-nq%o;$fx2}IIZgBN zmc4ml^Q|0l>k4(`LX7nU_s=ww*s%mEj{`r|N8wUODj3n(qz|IXX-p5^B1cf0Y9=>9 zR6HpuBbnGl;qhe>}^OEDp4P1Bu1EtAV2T;Qri zYAj63;`5fAxg;1XQ3nQQSueGfhY|JM<6C}AHG8SP&bM-`A|odr&zQ&UNVQnWr71@B zkVTl#+Lf(q5hNtd3kS@`T=BRzKKb%0y~$ETh|ZZvY6}5hbfk=WdbW=zM=4>(GYs*^ z+bK@?#di~C%hSbw8`a3mG7YS5AQZ`eUiDl9rE@r_RwwGI9*R%591AL-U_ar`N9k(# zD>-{3msu_)j`+9Sx=u%8UZU+bkiv<6SHoJ|BHKtDrRl^)EUIqo`YNncUD_+mC*Ng^PqJFLsA550BBwUL(ax_0&*Okw^_cYD1U1!jezU}!^;M8YKZ@m zn1*prVT}CM2TKm@4%U{sbOSy68sx^>QDo;S6g(gtsb_`=Xh2@$eM9XdA5XfVze``o zBF00NfH9{*_A~v6RdLFiL=VqlU_+&3mjRizHtFO{g4sO=r(C>@$@|DR@;mEylORwm z{foJSydLkbz+u{fJBHc3`=BEZeH8KYGWawT`l9?Y4~-zFntmW-MWKXBKi>X0Un@PepqaCWhu4RvX*t$* zAb9C?+oWJD$Db*A>*p&z6`k5<+aDd8O-h+6tSY6(J`Rx&ICmdAfJp1ic~EqCgi{be zyW21WK(7ZDKp_ShCjqc223TEKg)v_JP7=@glSQASo8RvgivWxGs1-C@dGprJZvXGI zHEM_6-26R}gH8beTm^EFfX^IO{8{juNlyfTp5FLTf?O2=>5*JG2P9RWRjIhdd1U@*BPe{QaPoi24QT*A5>$TFk71J z2E}7E-5#3D<)3_`IqL7K|aidGY${VbBN`=9KD#X6?cO zCQ6Efp|I6Wxv?<9WZT0<1;DBCH2Z)+TK#soZ7T?LPy{t!zm5JR&74PYUH$p|W@LZAT*Xt&Z~1VTE{!KJgC_k+E?W4ekca5#(u zr1@WXJD3R@o0~SAsm5Jle|yn@qL+QIHpogCWZU;@@dn%iFoa0#xQ*ov&phtyT?A#5sC7f-TFc*wGUbcY?caJd_HK)de* zsnsL({sh%?Me*%VtcZ$gp~wW%?f9n!+XJ|V-xbG)=GMIm2(4rJT_&U(@MQEuyg2m3 z1wQHe4=#5mL5sweofBU9`ewVMTz#-qzjypQ->C$=xH^f_(shGZEDaKM%BkweRS*0u z`dU9|3Dy?SqreOOh;-{16!GB6MjO4YW~??qNd!~{zLkbii4GrTjknvdEdMUnm5R`59A0wiF!qwOsD;w@bQ9!)W zco3I480Yr10|g^5ooyvNYqZl6%Imh#8laOBlIf)8i$V zVoUaF>r9-}eKWdGh;fiMN~ZnLq9+xB06XyZ3hF_T54M{8F3%Yy?IxW z1I(7^(5g!`+CUtmR6LzR@c0|L^V+xQ=N;kHW>nC0=D-Sq;OmvM-#t!FI%i@8pRcLK zOPCW26R^;`#BW)0exLGVCWjVVBV`SJ!#Qs=E3fBHl-)gbwYIS3ovL0b&Gx{4m+XYC z)A{#hq8rpU{2(e1xoGC;? z!~ZcOL5F;ZKaNeJW-bLO+Xd+*Ov8s_@$wo|yYKoNtp{#gI8+3$(<#b32#gr~Tbp^Qtj!7p@M58iY#LAkg}0W33+GMGbWz5#4k~1xa7{))09inrk#%DSf?hj3D*U9 z_no-mi>^6X1+jLe;LP9kzMf@2kqB46NpG&9SNi-{_&stPtN281U^zLugeoVPE`*jS zz3P01rm@RB_Rorm!*w#@QV?e^NrJ6d-I0Y#$w;Q})QMy}8k3r3Ax;xAOnm?7V}^Oh zeehywWNgJIyKsq>G~`rVLi-{3a}Wd1*nKmLeizZN5EeDeqc*1Cv7Tl%1j9nC4->zw z*;|q<#hQO|&{6z*EJ9}SQkqlB+4R>h?5Lsn5@Gsz^Q&b__&^;5W&7|k^KBJ%eH2N$ zN^m1*#lG0Rw&r+VUXHMcwc&)}!df?BA6%-bQ`^E4Lz9&eM5dk$bDzZg#;doOMA$&D zXrP(6d%|PI)Wkc|RZXKD`-YTJ9ND{7e#K#Cvu!{%wKl&%^sji`%8$PRV)cl~0t2ECYdVWyqYEcq*r>w~=Uhxt0jNA}uF+(0zIT*Tsc?g<-1j zPoY1j9sMwT90lGP7ZU?TY75)T)QAaXTJeQ3eX3IJ*vqU+?5nP)czli)Q|u_7zi(SButTffw>1auqjgnF?~Si(sx4Buapl}`*84Y($z`b~yi`(8#QR{fXEC7cx~cJbRb>Jc>LgVaf@f<(@6`Vo2T;f| zSV}0$&0~}&*t*Sz+Hi0!AD|K?-h6jTb5iqlOBs=%^F^A~zGh=?7hc5{$wYw(3_Yvy z;G$T*_Kzc4+w;0Z_W4dMQL1;I#H=0oo5TYHMCQfc-+NHlp)U^UHCo+&!>A4odvQBJ zJ`{=P>^+ zPUi0@Bp;{r^!KiPquPva?Fj-@{lqe7%1CE>CajrBdUwLh1zgEzxzWbdKBDn%{T&Ux z-w_}tB=XNc-lywFg&wA?KRs~5(W=EuXVMx%dKq-NexEcrLtP{@AF3dO);3KV_Tt{q zmQR|a0UrL}AS`{oO*!66ArFHKH2soLJ7@4%DxYoj$XEW)1(w>HjF=HLDGBX3jmWlj z60ie94O(=FHVB`s2-o;!=GDdR`EfXsoa{&!B@LLhSoVPOmp1rcTw%a2?2J%XCHG3t zn7U+1lJcxdSuh%0*)tM<(=gE!;V(w=jwWsdZ+rG#sBb*0mNwfEEMqMp$l242JQQveGkW__6K!wo`Tjp%f5zcqm%C}>$)soo@lyOwDV!7AaNwU2#+l7&%ISxi?;r( zm_=?ns9t^&pNaI7=UNxhv&6%rGD+e>usE@#-4Nlw`x6q=K}akv=t+eR8-xJ)bcVyV zbGCS`SV|@y>X~%(dl)Pr_(pcPNPQqG4Gs?}a5p`p$(48(OjE4Dqy>+iw77DQAkYYt6v3(^xpWaEpp1$XnL7Cr&{IdC_j%SkX z`}V5w?`>^ZpY;`lyS)xpU`|By{!a^Fgxx>V^mT%hk&)+jMSxgB&Y9J|wO#YW6S~^( z8VLXH7c$Ok`doCk-jYw+P-VTFpdqkkn7Hbb>}Q|d-wdLaG57>CX%!*BP-g0dEY3PR zWmo%@z9#qTUQqjp-x+6ea;$Uigz)d?yo1-NLjH-#=6e3m5MjKTT6SpJ|0Z?Wol?JX znxok~Qe0?|(hhXqsl~S7#(qcSdB%_$&o~ier*JEr?MNfc@GpFczEd;WG0$m}9EJid z`*HM0J1ObDkFOzpb8;6hjKiBup`ULR1B2;etyXpLsLH+JJV8!tJ%*~O_D=JE*8!qT zU=o|_#~O>Om2+~rFt?-lYQ1n_5&{NaKNB zk;}g^8oMlOD4Ha2clGHxbZobyv$M!JfefSeXTuqJl>jbi#EVnz)x1T}Kc^RUeGQQ5 zc-kLwj;0oeslw4nmW8cX*?gQ?`zycr&xwa}2(MAVbT_;`2QAN3xtH4l1zl7j=Urty z0_I)z$P@CubE5hGT{&h<^^wWp%duVEix*j}`U`Gwlvp~52(;&`GLOMa15D@>uaTEC z{hmPiJ0mA5MU&AoOda8oY1BcwYya69ap-vPnIJmV*Md_!2#98^iREu|?M$CyM2J}V zV-Vzxkxp%%5s{%=6PHA~;c8bJB+L5%7 zO=(8b@M3vY^x?YR$rp~bxg>A(GU=u5Bp~I*Q;qTe{a_}u)+dl}O0EqE5!(*#tjJ>c zQ;NPvx#i|l7>~NzuojZNc8fxEB_eJC&x3QiNebj`INvBz`$8X-&5^{cF7zG^PK8jTH-5<^nr zPWP`}uZuxzOP}4drTS{UZ_M+&?z!Seu8gB?I(xMs_XL`+ku7F&-St2AD0{CxH=^Ct zQ=U@ra9@d}KE-J0oeEru@`$o*VRT5xtNQTQhb!6%{Dp4(yP2ftw|0jrG^ge;>A(WJ z`^CC6uMh=|V?3j1F9x&5tl6NM;S+J_wDQ!TLHi*Ldo=ND2tZxxPg`!64A zok}0p6E=vl_Rf7B{B)G5nj)(A%yJ&wcgfJ(;uj7=kMR6`BqrQ#?lgl4)onw>k2P@k zd4GhaEiKvUzeSfQMyo*E4vbjISGD@O8s~mxd9qRadoGJ!Ic`zm zl7}UW!oSjAO*r*$EbiQ{J*W2^EL~(}FwAIKx4Jc5DSI=Z{YSfs$OG`P;h3M_f7QlR zJKQxN@YFW`#K(1UDu_0~4Yiwf?P$tA#0FpO_~yy?`LN?Jn#pvLb!k0?LdY2(rh5#^ zB)Qt2;H%lCnemPtKecA|8k`#0#d-DLti&?3!1X3qSa5$k=R+SNM1Rem$W>=ta;sGl z#i{?>iQC#7&oWE1B|vUcPE%#;vF>^b(S6?PNsZa-YVYQS;naO!J zr;+WTO$@lLu5RJiq-2ascTPQ*ZzujBX|X%98R#j08|)Goyp%_P&YhC2KJiNRSl(on z#`)EpbiRwQH(U=W5qScQ1R`p)Q);atyY3=|8Xom~Ycw93wJ1AZM$J?l^!grlbyhr9 zY`;ixztySe!{V8A^Z#>DqmW z8KtFERV!(zmci^+u`@|b<>#}{;6t&t@R996@sYETn+P`0u*Cq zO6?maa|!wc_Ze*B^k5o1`JHdYV)$(Fz%_NeI<42a@3Hew-XJk*OKBIw_(pT z_}(n+ntVK&Jro^(c_|BNX=T2g^oqs|X`JzGKvzYF><^N}e_8jXW@7YDP0fRAJ}jLl z86uP-VpZ#X{xgs2J20@*6+e|F!DCHF{0RM)=)Xg%YGad&+?DNI|p;UNS#m( zw0IUBR84;vc{d#PSrsO`j+wwP!M`?TySxMr`y&ceX|J@s(H$`ccj$23<}w0yH?sks z%(v>L3u>nMsg**Gum~uC_*~2@sKDi9Z$Xo#vTkj<4Nq@{CC z#hpxtbtbyPgDaL*+v2D(JUl6(uz<_9QV>17I^jiHHcw+`^ZjtdCv@qQg@ayyzejkp z^-5%eB~Hq^nQbS<_NZC+dI8LLLH)Obz7&h8o(VLAiuC?=97xdZtczLiLCm?+S;evsd_VfFvI&a;xVvrpGCA^DsY7BY0r zrpZGug$tBosI%|$wsxNJ_T0L*+nL1}>fA8I;&Q`Mhehs8`w!KKFY>_nS*B{nR~sH5chTNM!4-4pw?$-?E+NfzcA> zU|)Y08tl1TbghfcKa6@oeMPAX^jQC`c*sXa%3S*tmU&4`>SNuGY#ilZ8a=8B1mCox zK$#4WjEl)BIlFi2`@XFYhHsuOZ69+&&HCqX72#m=2@O83E+z4Mn6r=?H-@0iRZiI3 z8kXmzn@`l51BnQjirxbgDIryB7x9L)M2f;{H&^@;!o7y?FKagy$FQI6EerjO=aYQ;wF~EyVZB)aw_A3HR%(w z^h^84X07a(sZ}kXHI&#h;}GM2>biFEK6T84g}ACbopG;fCZ}_)MBvIaL&HC^WgeNZ zMjN|<{m4YGe*z72bTPIdu9%=*D3mSo3(V-E9VKMUb@BSw&Krb7VNo(#0t#&D&HA43 z0CjWG(XW5UbLy~i1qJ|p;)k5F&zjbrY9ybEh5L^pjNO3J=#=A`BoQklLaa*LGA^qr z`=upsb!Zn3Q`0UW-^G^b3Y7@*D#}N$O>6X2D@a`quX?TTR@a{+W;3j=B!XRl$feh`PV)%vuCW ze_DdTxUeq)d>+6Fedcj>S#35vPDWxWO=u|%3$-x0Y5$P|G=!dv@5Utt%s2WUh5@tevwqxTo-{wL{pt)>4Oh0q%yI<|Em91yBhE{Ao9i3ItEYe&rCLYM@RBCh>%l= zMoA+-w2w~I%46r{s9PQ!Z^_7i_nZrzZucVBkhFCA&h2ey=MXEJlJEbc>8it`YM=I@ zkw&_^ySqzTx}>{1rMnxXkw&^3q`MK2MnF=K4yo_>`unbH|JvQN`_#lUGxt5Ad@uk? zI7+6SV2Gj^a6Zs{H^AFimldaZDYp|g&oz@)!!;^Su5B7#!GT@nJ0-ki3o6D^TqV$+ zAY#^A=n^?jKmiOTdu6M}PIj3=C~tOiHuHIO8+yhbttwvj?4HT+c4{j4l}>Lo2&8$m z0?+{JJoWC`C%Jgzo09*JwdZ;q+(FCCs-&)3|N zV^qTiMuj)J4BmUCqG8#T_mUemAS+gIM#21rLy&13A;*+RH{D#OrYzDBzj%!X+M}B` z=iGqDT_eX&V$bZ;1KF1C*UulY`NmTPm0JS}FD9%1q=48XmnR;>;Y|rLPI=F!R;Fb+ zy0p;?xv}3g3ncA|0pc`>e&uU=7`tCLu-T-8GG5i~AOep%Oo2Ku!8e}KKI7@*ubpqW z8THZ4@OvGklpU|_Q?b#|W$=u}QM|Bfy?wxb|HsPPb)DRM`^|`v1>)Rlfl!%C?T7sU zjkCJB@vvNjzCUzQ3mAa#jN!}7;t+T;QsTigTey1~`UH{9sU-Y`=r_alALO9Zs@IH#K zH299>H~M$AphGpYL<9@?53WVFX732wh!7 zO)lljO}FN!B)N9H)jzA!8qV)lS)6prbg^%hB4o7o{5p#7Es2OA0nql;zMtFQ^*|*? zA`<-O@-wyN)m!n&N$prB%+S3NrylcJ;oy^NsxxqI+!7mK|h$se*a72^IUW_&as2ysC#y z`|i||MWdkVV^`N!$fo1>U;n#I_=Dwznm&p`Fq%o^`vLYOUXAw$t^9)2u{>|<=}h`T zK_0=MkqQ!)*zneHn>ywMmq1X*_L?7?-wC{dC8Rr!X$%t~5d|ZLOpeSZr>3m@D5Te-ImTDjbSZi`l}#}M)r)F>*4FAXlU$e|k*RhO+U&V1 z676XW+NbcF=jPhe)x6j+aEmo^av@d<>awZvrpt8%fI$}zNtI8F8-gD~r_gtY@n?h# zJMaTyjolb7g>5I)(K=s`orumG`;hM_!5ZV%uCg|&V5AO>P z_s!D&_ix9kKgAnR^|2_3RqBd&w$W{%&$~uGylEnLX2Y_XdJ~PmqV`Q_+&t;aF>z!e zyLzriL_$W90C;J29C4Rzwc;YSw?vC@M5%TLH$hHIk+cYF13k8`GT_^8&(7=L zVu4nU^QLc2tRXeZzsA%}A}|Ka z5#SOjmKChu43e_)ihk6$i95M0egiFDr$$rPJ%%{GJ3lzVXhwNcn_Xlk+D>w!=JWh}weH2$+bvqC9`znhX?60@25cuC)w z0=zMPT)C9{IY&@PEuYX`cV`YUP}ih*_(>zo(0;oOpFgcomf zE_%@>%OB05mD&pao^|JoKfidW@n?YEs6_wB!WOAmP%mqSxA~|40yiOaRCab+ib7;$ zJ)Q>?#FERy7KNWMioZSAn6z<(ei|LluDM5mafIE^_oie}5Ze_MyZRdB^6{nxs$z0f=@Vc9jB38>(}+gZn>rj&PXw%-o17Y~p<_TO8Fl3^CIs|rK-N662*o)w@?_$B=fXFA-p^mh9!K>Je>eNx|BkGG82>(YVHluVx3gIxRMzxNCknbg z^_j<@-k)z9QI5EV3Is<3o4GI(x=w0;@Pql42CBu?z9(GO8&iG`6h?fj)1yQJ-p`<) ziQdWP0tw_S-hzY9!sQQzSh+}qU@-Zu^P%M|wB_|v{ncxGM_c}oRf&50_^sj4W_Mm@ zSJUChO2@#^r4^y=Z_g}PZZz1c=P=|ODoq~I{AQbf@!fYbta3_4XU3aq9k%#abY6_# zVpN^qzllD1@G37>gCKHWyzv$E>{?}Ff5?yjRIx2|U`mmRXEWg48<#zPi&(w-mJJ24OARh6 zSGt|AFTDbDay#7~as`s{F{}5!=8VkV&AWxDt0GlJ5)WpCpka@0t?^s1*aVGA`>&W1 zE9}F{ig>RV)4W0=zzNK`yI!O{<8oMj9Q;l$h2-#>;(_WAO_*PD3=Md^2Li)t#vXO( z6a0G?u0)0=@vN1G8CVvVgQcv<{^|cj7U{7U$4Exa(+#Qb<&YDO;LR}Cu=}^us8Lso zf~isI;*7g*Pnpnb@^~0sd6cY{yR75KuT6vpQRhWs!r$dt5WB%JjVSs@osHb0atDHJn|2=n`)8!wzB(1(HaEc}#)g1Bq%J+A{ZmSu3~`O$I& zBrrUdp7GaN+6x9gSWQhIP30L}zh!{iP%TOcN z9S%g%?d76RsiZ~QKK9Sjmgi~}&mM-t?OYvJoYqVfen6(ep>L#bi!?L!ZfNf9PJ#nw z;h6ib6F9mSTyi(7?^o)L1H;gcM+X50U5F$EAW)KP48aklqL?RfgqBogF|S)N9ya;o5>`qwFJ%!knJufH)&j?2_v^^Gh`aFG~Rs%Z#~1QF4)icQ?BhNpkBG z8~#(q896VvzlS#8QSt`;l1C^hNGlCroLS-FM;3UkNjzy5z%?QZnKP%s0r93;TagCA=|$MTEf- z`T>ZN>2x|NM;Y<3RLnu+zZy>J#jb7g4b_Sq?7c{n41Ywt%bsu-)9su_pY9U}hd9ll zoeYcal06J^hKq$Qd4H<8`=yQ6HweXx80ecl29`jR z+D#p8ibWKO6n>7tnD@2QBdrkkfJGgeI8i25$PlMU%Y+eI=yhb1of8+F{87Hm?+hjh zrb38xx8zP!FSc6|C6jzA5pC`szvx%tZ`27mT)e1CNMy~SsQ>Npq($UxhZMLs@V?3e{A#+9U*a$Z;0~cTS~&Vh!c3IxtXK(q>>4e@e+~HZ==aw z8}WQtlBZ0ny=t88NBGH;9XJ+}eryw+q&CyDV6^Nmgkg40;J>@al_{ov}PIo0z z(q+fP!7bT5iSSQZbV$N*oh&_MOV~yf*RHwDk9@=N+o$KW-2^|Yc_=tQP8RmWO}=#k zM&$`?$qwuK4n?hn=p|bM-W|K@hFLtiaoYD2tag*47-y12_1>mM-FjL1s%vs5Qot{X zlwC?sO#1UJLheX|Xgxnc#8rh^|2oApdN>e%x990Y4hQgUi3eg$ zy5_i*rmubX-mO}nsw`;uPX2crr*|AFzR$NeFdL_94>(6AzT3B!ZPfH|t?Q5hfI0<4 zwJI|vnD?i6SKDaDk@Fb3a18GzIPYas*1kDL4r=`;eaI$yPR70nV^O5-=#rBhY=f8C z?;`C3%?aq|M#qk7%iLJfo6y9aPc!;W>g{~38j#Tcko6wNrhQq(`8xt78}ESINd;z2 zYYxfp8i>%OLa}XQMsuIEhy9_QL>ck&=ex#g%({w#>Y)LXG&Jg~7k#hDH0UJSP+WMp z7WX>eU>wrDrI|N;{|%;-7DmMt0782k{C}nKHL%G_^7mTuz)i5XSovj@);B3EqFL8W zILt|m^ir!r0KLh#_hi^mRV|o<(Q1E+Sx))6@sFR>$MnyQE-Va>n!d3AJ$bqM#e|xu z#(n%_LbAw41d+VRb%W@=KcdnD3oM&X+2@l-@?%YjEC=<%DTGGpZxZZn6~M)vL->ta zayzH~(074r;ytcf+P92Xb&IK9a z4*U-9R1P;aQye8^FOORL&*!x!15}~TtPrX0uGf{j#dGo+ms38BI*WcMBa5*2Q_$}7 z=Lk@Yp=gk{hoo^NW_f0HlxgyOW2P4wQ&=rI)q++yg&Vo2mvEV)dI#2XfUT9F^+ZQ> z(}1&FK^)@m^?4W;p=u*&lwhxJ%RL1D33TyMe53YJBjotjF_9I^I?BgYBy>5PA7o1N zB1>fA6ZWxl4wk&_EC^fVx>lT03#BlnM|y|&uyQSmI#HO4dbek8Pn^hz4+MXw`Av{W zjuN@f!jk>#jOj0fw=zJ<3d`b`haw}bz$1z>Z&wPldDh=1S^2yH4UL` zxWdv2(Q~d@x}?fk?8!OZX9J^atA2Fh5?UZkX}KdXx-&ivp@<-gh@d_^bX!d4IeMRKk)Q(U^ZF*&*v4`l*Dp!7UG`}fdB;B zdW7h+scbTStYGTNhnA%2FRc{eV)j`e*qGds<8F#czw^|CoU$B)9%5M%{No+{*P*pb zv&;kR*ZB8qKhVg;;iK@X_T^uDc?);45Z}w12G`5(J?gbBhA>IteIxvpF(|Z@0Gxbh;1kEGM3jf}w=F^E1`EY~+sB2qPwy zf7r|*!gmhA@eYYD9VvZ9AGE<_e+6w+Ra}h%&!Cel#JWq1n`+ACCVw?7syQ4UDX8y4 zGUvdIgOQf6bc22V+Bz6vCp8gUS~SN;DvPA4Jj`3S#;}KM<#P2NQ`I%W&MY{k6k2pZ zEOh;5g^1lOUdH)`xfn(%mpe*JEWE|Ne#x=ZGg>}_Mk16J-aXopMjXD~Q-a;w61vkR za3X)udCW|;;-|(E_jDpGj7MKOCt5**A>O1xr#yC1o%Q;qekjvTzUg)pq=k;ilJ&=U z>%2V_z?l8OI59fp&qs5eAJDpLh)t&w@#*wf2$$OaC(N=$#^g zG9%fEBeV;bO+q6+XzBS8r4u7Q?(`Pxq>T?BQ;gaRav{9D6R$F?L>V9KJ%{(4@tl0Z z)sguOZAFwv*qnbpc#)8iTK4t|Y$1O5W`czrNfj}!e>+l(P$=sh@*c14Nrj>kI?$;{ z*hom!pbrUK9Nv47tr!2Ta=-a3P3eO;3lX-NJwv;|%PKc2`uroB5O1@(1Lcvm{?@Fb z6+2m&;B0Pv42&LD=XyRhZT^TX*7@Ny+ z?X}s8vqqo#k{vv|ROY4>1DGEH01zS{V_6K=ghiIlK8=^bCRyW(OvD&jg`bbjx}r=u z$%Ye}8{)s3Tr~66C7z@!E6_=W{4NoN@bDVd^uc*b*F_B zK@!d^;hdiqJ=jY&?CmUNbd%EN$3IWQqr0@#KmqWcLSHvEh)3BFv5Eqt)5EWPq*K~g zCe`5dx_ui`>b|{N1;PNb!2H9j6;E<^glAv6hwkuy)Pp4qAa}8^tg!e&$rt<6z?SHc0}+R$u|taU6p_t6{((L#g{nf{iTnl<^&u-y(S2iWBg(|wQ`=}IUD zmS?9+gp5BW6Sd$PZBV3tvzCPMZQX^N74DBJ>St&rTmiE@;l);g!?~nm)T$fgqj#-N zbFbGE4SpfJV=Nlt_6!M3Q)=<@)FYLlwu;pP;zeA&f6A>^QpOOMWwyR|JgWqTdIa9A zSmuMBm_v5kz>eW4_?+;-Z~}=4h?@xx4yKTa?|QynOAx$_e55EYDXBK-{)_uWMRlC+ zWY+I++Z#;+@h5r>W@Eor8i`~Gz}C-OJ4xu`M6nw=ZansfXvFf{%>|QHLKcY%E}nKs zOSX04P(UKbEKT^6mwuhK(VD7nRwIMF0HiX0ySnXM zPczo_lY?zpJT8HArq_@V#+`x;oX+y308EqNft=hSDY_i6_&iX|*=$JkHrkz$g(D|c zfmDE52j$|}==Rj`{%d}{o67We8pw@v-_I%BdxV>Im1b6BXLN)>?7H!vmMW|$Rv#R) zGzZ439=LM9uv^NzX4_@R-qY4Cr6Pt3kTV)p=o$^^df3lqBK`s5jP#nUk~ljcfojdi zd3hEHQBcgmp4JHhacBsN*Q>|=)9ij+22vy`nfiFnh9e59SCpOr@Hwm*lacXsDzY|w zOmwIn(S%S-1$u2eiudVgQ-I=oGG7C9T8Vute6nROK2>Q9HBB}IsVd5y$JLY+rG^A2 z@l$|^<{Od(^P>?&0NUt1p$=EZ>FPM7?#@|!I2Bqwbp`E$PgF)?f7{sRO*Hl0!y#0X zjain!;C6+00YxYjW3JNOqLv{wCTCs_A#A=k2JZ=w`fe~~&G znMTjPQ<7pZa?w43WYnrQM-u$LAq&q%nFomaW3Ko@F>JOU(Mm%c8HD+qtw3vwS02O_ zh`#wj)(@hAPHubvT3Xt=k6*NE4Uh)T1z$nLo29wAxAS=ljq*Gkh`WrhO;1mE_+Iw| z1gL{@4AMiKEtH`SggTW+qg{s5$dcj{#>tr!V)gsABWW-1QvgV0KyV6DI^}DVJtQKa z7n8+ONMSdD!FV=A`DLsck6VPd7TMWzfm1PYj#O5+V)H20ePjfhR1AhO$lMRFe%>cs znpY^LNT&u=Mig+9M!EHQ=+a7>h`Y&SpQx&*n?onIPtkuN=L@tS#kW0gHz!y8$&1~Y zQ>GAu!8sA@3)TU2X&4Qp&c=-M1p4I<8e%{SK{Nn2?_InGzqf6Yi&q2BodU=#q!Fes#BLEXRY{Q{o& zE#IptkhR94_woaqfX{a#)FFle11=x6HWmVo*E%r>6(Mk?11|L62J(= zBSOKUG=01&`D8FYHI+7zUD^Dh$Jcu}7B2#YNVm?Y4`grwp^58+Pv8OqPS1aCnDknB z?M30+LFl5Inws94eJE|-!O8J)s$7J1NBPvGG*;k;THA^*&aLzx7)2EouhbyXf%jT^ z6WNQv2Q^buMMYQW^VS{!Ajz47WQ|+;gpbb)#EqF%e6c>SsSG%x_;}a#RB1~H;9md9 z!p~i>=u(0B3=QP*K-H1Lb8)4=rs5;f1AKm8N{20y(o0LJtfa#lHp+fYHUvvJ+eY&8w%hH2%zF4}ugDfQ|@RxC1e`3e=yqXmgdc)QaH#vXT@? z!^Oq11MW?eb`DiIBGx{eyqaZ|3%2d&{3#QRnfHB5T&#d@-T_R|tmr$m(5x)oyk)uC zTInJ-HRC;8_w>So(qU-8qH*}*c6o*(Xya~IrCGubb#Bv~MeIGJ?tKi40n{-v z5RZ&P1ttNEU=tt#uTT|$b*;3@YCulSFluDlx8@^m)$E%9?(F8hItH3b4Dv!^am3}TcXef1s zMMXy;`f>*nG37NS z#KdItxt1P`$iTydu*sS!s8$AW3_WNtz%>LM4h{29F#UNM<~ZupC-HN@c5RW?e!)=w z(0X0$ST9FYeF)7mydtIpDTlp>G&%0ZhjtYZqEiIeB6w>(x2_2?2Fa>lTziOAPaYAY z$kYji#s)pQ==bProlxE;P@#HQ`Zvxn3ax%)2Gy>BPsAHQo$y<;_u%F`Mnl0#qp(D@ zo;#4k#o<~^C)4jo^XlqqcsQ(eHul`k@3^?QzZ8KWW2UUGjuJB)8#-W93gzM43i&Za^r2UJd`+y^hlNAZBAnxC%E-&@1i9PNf=^=w$_ zlV%h9PsP@U2p#y-F~s(ekfLae^GX`3${H`cSS-X6(raOgQtXb?73FEvIurFcvN$}8 z-i~d1(To%nFqP~ZIixFQx39!rU$MQFheYPkCZhyhv2bIM{sI;_S*pgc`#pYtK7a~g zbbUCNhlEZnXoUZ$k+dpo%TmtON-bXuGRn$wGSv$iywjh`&zzdM+RSE#@2xRl-=@4n!|eA& zS5+3gko@pr9qcvG7yvdBdc1=Y1p)maXZ|Nf9h2o`a~<8U70npmr5+w0LST{k!FDqs zr;1W8zZB2-&r6UwnaQw6kcX!g`8O2VH#YB{;DUr1+qEFg)A=1~foGGb=n#LDpEP#y zyE;6@)zqbl$}ywG>IYGQ(3bTvxxrTKG%@X8Vie^^HpsCgEn8SLP-|rPPFwb*6LJ1D zLfLKggol{6y6Co-rJS7(*sdKcR+YSmVZJ347Z(P>Ko3_3Tdp%gAoIpopBo!DH#ae{ zAn^hxJ8qC}TBp%s#oiZWjsz(jgiw_^LGKGBCtX@z#=*i$`}FC=t)s22?di#v@i-FA zW#|ei-Ru*$qJQDwgLGcP1z$JBY|4hC97WvS>I@CeGW;dvhOLv}!;$7E9J&#zOrkG* z+9EAKt(!X?%m@^e5ZGcBC_yhgH`_LbU1237QCvwq$tCx_ zZK&KdyI^pPP()%pnh^vr(QvAn&%wx81+wOyx-y5pFPBH9unXCK@!p;M5_bDV%(>{mF3;Iw2r_tL?F z`1j3j+Ywip;43z+RJZ8;SlB`w<8&Va5fLTWIiX@VF=QeGX{4>2VE3?{}M=%Lf+ zYLuR$GT>^ya-g7xC(ibOknd1_K)ICfqw9A~;*omu--k)W$h=SG00JpywQ42HiCd}tO8 zkx5zCm@}2_^_~-)-Xc#Vb14_Aac7N#`rn+>JDd^vnxn z#mZAR>SpaO((@EI`Eqi>9sIUBSxkq6txNv+?!SeHoMfgphJR}5oCyx(#>T(7lU-9w zh4fYR=xY^xDxVC6=!#)2hThOZ2G@_^YdD1g4!+bGRXM4F)T+HtCLS&Tze`{qJofH< zl?XD}+=>wVTJ;ik)g_wH=L5BN)}G7R$r>|tay}P^yDOyWlT4TDjRZJsz~%ZAPO9U@ zWoo(K-%PPM1a>v)cHb)9CP>P#Am2_iLDO2!A!+St#%`$mGkMY8rxpyc8x+GaVCj;a zFGY+EU$*453c0hTQsFW{ZNvorn~UA4(=8Y{e{sN(ztt@qDf#}aQVP+YjHMJ0Qe~K< zRl40-D>m1tD;^j@h-S3U{m#9D$uwH^K1-Uy$wP3H`LF|>qQLraasMK~3C4_-B^ivI zx_+A`w}5l@A7XR5*PMQ4m+LM%gEj+Aejl`Bp!u^%;H!(> zf8(?K^f}m+qm60TkF|7;A!`)(5HF05af>#xh0#`@KITDbA=^QLQxmc`_PG0*KpCdO zG{?xr@FrE7HNH{}LoAYQ@xIoS*e0T0{%$hFa%6k9?Dpg zuRC@KDz;kPmvRm)at+n?xR}R%U~LKbgp1ZA*g5-56bvRMBnTU?S4B|sd&)i^RgpX% zK!`(k!U;!!bq(-jhN>-fn(UXV7^ZK67Q2{NKOh1te|AWeXB=Pqk!=i?7vCq>KBftu zn$Yg{+AW8}GRI4V;{AE6x!s9d+S(Q%JFL~6Ox!P!AG zxxRn#-OkhaX{&}`T4m>)wJ7{@ zNrLk!p+IZYd7|=F^$BODa}XX<`#_6|fFY8!nAfcMa5Q@t$hkQa_q=Mw?D|pJ zh{$c|HGZU56qIW^U8S*Vd@J-6^OsIe32Lt<<8gT?fC!wPZZFhJ>fJe~v-l z?z1B^ao=G7`*AC@GGp2L;Iei?>`CDBw!H<*+y)*Rp#+{*mDpR7VICpZ8xo+G->jGs zk1;}fWO2`PZC5Wo-I`WKK)t5S(lX~~Z#Xxhh_A(_=`v5#J$4qD!e&Yf`)@BXyNYd( zIMHZ^#?6S~v^lTLc!WO0s^gA1vm=#syB+8gjj+QZLZ|7-aLC3#P~L@=u5We9-0y33 z;>-Oq#QI=SH;-&Ci8dd5yPL+|$JuixW1WsN!ATklW{q_#!q?Y_T8*Asu27Mb@pS`d8{zoyc3DlYNz1&;Wh4mK7 z8xKjyGG&Xt6b`5d1ORDtt?REm1y=WEoc3c`y=~plzo%)mjXEZnI-7~JLR+n?BVgeR zI^S<~XI=h|GVHA5Na@vNX)F(k&;{Guc{u%8tyh2Nhd1W?pH?Asdm!xKNH7$zkl>w5 zE6BZ*c`Im@UR~HFX<@a4MJ4bH^w zp(&r^+dHZ$JjRnJrU<%&6q~PQsDs1TzjC;!T?q%xL`N6(1*;V-DKUzZk$x(Fcs{_= z`M|RAK=^Dv_R(5+H*na{0BkVDF^0%HL8oxtQa{=%AT>htM=S8$@EVdyFMMJQeCu>AtehVzgY~73RJagIM0<(|5to*I~ z=JJ;eKS=A%TTv`}op#H!%wO>GH)~=$?9e^3#QoQ zLP`~3WI?~>P^fBr&`af=kR!vjg7Wcj9-X<^AK^hqKt6J%u@YUw{*oJy6uG)>65RJG_PQl>T$dc$mGO|MTf`_@P4DG;(i$rqjg33{$@}V@SUOn|4t7af&8@fdOW{2yW;qe0Gkykt+7=XB^=jg{t zHV80Mr|XVk?NgWC+WkC?;h{gZWawXWy+Oh$bY~0VV|$`!YFy@wkTcAJb*yv0D#zK} zTfg6pB3XR!P`muuWxNnpHlzpb3-i}S5T`AdwKO}ydq^*@zpjtc)|^v8RCIBIuj?hW zt?8+~L9EW6?!iplpM4tG2fW4&ik!K}8rghq2@@Op3#o5iGd;p3A+5tr}7 zm3y-2rnD;Dr``T@na1#4*H(>Brz-2cU+9t5_rb}Zv=5^%Np70*ahEF@|6mAWIG#RPE#ukeX zWh1d%S|i-*G0H>MzjiKi(^=!VG`a(hhFrJ>=lCrvAK)RCp~;qfEDJME3U8JoxCzTv ziX@JS=ksC`PNU^`$G?4&%(mTVO|r>9L~u$L0Br!~-Up_4j23m+>b}PrEh-m(-ZfG^ zr&KdoP-f<__OL16RnnC8)So@F%sl)1+$`5%YhgPF(~aNO2u*mf%12NP@o75`(_1$} zjyV}rnvgPWC0YnrZ*;qv>Lk~%U)>rE{4F#f!%V)GR3eG35ZJjt{!?gqXY)8W;^F-n z0=aA_ZZW=-H%p~kYk;Sk(R(3fm_5Q+4lwd=)OFz@nGitDD;C-lG|-(?Xj+ip?clyz9#?8&SDXw$q(OH?YSMjpmW8}YA%OuhtjH)=vg#m zr}M6tO#JWBqy#Hxy}D}`j7CYcg93JrWp);@57Z%gt85Og2E-BnIT-Kw%g-*P*C&-J zMxZ+=&PHL1mO`r1Qrjk#-VOZQE?MgS?i{BXOY|bh4RlW*p4`(-b<9HpXyD%uz*p|| z3t8Vb(vOImy`06WopKZ6ssF#t%^BZ_y{2C(0#52WJ~9|@fT>R3P&LFV@*y#0H&JRFfNU%oB>r21&S)Af-(FM!G4qktX*sL-x9C7>|2 zhPAaF%y6Gm13pV9A68g2!z~JV1*-(j7!|1eU9#_BYUx}Ow$~hHnw|O2;yoW#!*@`D zquN{^4!4>(Zea=EKg$Qivn}FARYlQ4Y(LiJ5foATqY2$qFxoinj&;(%3beeOPfn(l zVIEn`^BojD!u)TUMq}*Sr^3slK*R_&E(65pJdN6@uSH2JXUo?e?fZIAKk)Hjfb-sS z(P{bxU^`${3>;bq?zj@&{E_@lK}DNWrV#=M&wQDd`5br(YjYQX&&oGq|IfNXT+l09e%2Z#N6^SL47;&4O9ehtAb{89nkW?@aK(eLE^G0nYP=w! z1VWRj=tVY-L0G8j*uku;_R=Xg=G@1S;zu#piMyn~YUa_++$!?_W^n!lD}qs%(zx^_ zY)|Lsqml;?XqjDnrR~gm<0_W-%SOh)1;>~|g_A9Ps2H{4Do<9&2zse99t!XU*QdCm za=L?`3&?omDw_w-eCEvwrt_pODS>#ijTchkrC3s?*jICDU3mv>~FK1yrC0G3$RfDvU5C&($t z){5Efi^bjR7vd54&pXEFjyHuHZ0=A`%Va&kw7#x?WG!FKQea0vv(%V=?T6!Gi&j+G z-HOeA1E7{LWu{y}0*8@L&G#z{oi&gE#gB9

wZs)8?vGeUdAqu!aAI&S9Cqi=g&gPJD zvpLQ=scicgT&zBx*e?Qho@N-vfT$x_5>Yk|jvuwikvy0bO&mJ==&EU6lA=gE;}#lN z*LZ;0cv)@}*o^M14pJe~^vqXzUi8CS8`3X^W5d7(qRnLEV%ZJvD!=b%(9;;+*%r6g z#F3=Z{LeECU(csd2FJ{Ru|HL_t~0IUppzOu1tCgQCc@CuBrjn0 z&h!U~D#_Fm)*6g9EIGqZlRo@Vh`^@pbkWhTR_IQ@L`Q|oBq!^DH(YoA*SFdAJ;QX*TF2V4zO#!=T4|#1s z=83^2=lf@E?5{#5gSi^797+3aVete8NkgZRgbyZ9-KHnmMI8vtCT3@y&9Pu5R`bqq zSC_IEX*RiI#)prAaMCh+@iDvx=``vz`|VF|x={ao4YScF5znkm_rqoc=0e9i_rEK0 zwc|>oo(b-?65Ya)ypJvH#NIFNUakwz7Y++KD3x*O6w^?d6dUsfTxY~;erL(t$Ai!Z z;C=Zxqq_O+ELJ_F;Yj&Jn-$>2cCoqfBV-A)78W3e*Oo!`?k80BB-uQ74Q=mLUyLKJ01L4vH;nquRkV1lLg-GiWS#sbmNb`f#T-ptEek&l-5 ziKgR(&hmUX5P*AZxN!Hdl=8Ih)xgn1)Nr)8MtZa2w}Ac#9uMn03S}rpj5<^gZ+gSK z6Y#1fRQB#sRbYG0mY|QUc?tV}){ufB2LQ`mt=R{{I=1zfOQSS^z@+*&9MM1D0Ow-Y z>K8TAGkB4|%EwDT4;U!Eeq>LVtt^1-m9Um4pSrne;gOfCwha{Aqx-$DR}FxuKyn13x9-VSB~*+l^>K_$lj3^5x* zmk*rK6T@ExDUKAO_4{)~0N7I@^nD~5@S`C;y8!6_XR+~&*?~ZKK9#zlGXxjJ-{g{i z9~d5$yd2~8(1vv88WPfwh$aYb71lfm!u!vQHA4&3$yBHar?edn*~tAcxMs{mwD;!e zqv+o!Q6Vzs7GKDfs!StkxBtmFf21~5aRf{U20ZY;HRSy~h7@NFh)P2wx~{l2{yyOn zc6Rg;jA!_N=aZ67=TF%u=eIQ$hyNApgN~H77o_(`=*NJNuCCOCn~kY(-qua@f5z>d z2|QTzc|?4q=q3Rm6rOq_r387(tnIVDzynDxEb-r*jsAv3;oWK7CY|2rP^lNXZn{!1 zng+`wFn)S@L$}xNU*#r)g~$Ng#LwzQnU9yhL&81>Z%UzX6-88v>R1M4a&b5>70|2JjfVwi#57`F zCXegwhudkx`ncrOVu=X>z?^`TZ>fw;+{Y9^<|=S7Q}-`e#%69a_9gQ}7gz5NZlnqT zU`k?GPtz$%_>A;!tSDq4O@D*mxqZi{P9SQmIP}E*slA#33K$UP&|iL|*2#*G*FH}w z$~w6F0I&l^fT|gRqz`+<}xg2q$;9m?yT(?6MxHkBTwGyD(~TLg2z%9lKcar z1tkiMvCgs&XC-E165hX}=y~-ia>^UFJPlA9U$dNAq~E@0KMnX(hgt5P`y<`snD2P2`_5wO8*dNx=oSrL9xkA zRPJ=Kg_+f~2bhd!gY?>FXzCvwe{%E=s}st}!m02n7t_mF^I zkrC}P1wvdI;;~jYE-8t;UsJQ+3OemtkA%sxI5;9~=PLm-g zL*`OKnde!Qq73DjGQ^1_W5_I2l7vi|6_R<(JihC9&h=i``|bVk|MLHRQP;&|+t0o4 zz3zLjwU^jQiQciF%SWUKsJIk@Ke3!`*R#1?kP)|76Jlt*H^q{Mz=>sx>OlWN^O{Tf zN(^QZF}ofsUyYn~;xza0a1{DJ19IFxwpZ1RZXn{4tp#I z4n#*scYsXZ+1ZIRRihGzRP7zoc0a1VCZA?zVp?08ic3gHh>BYJESNJ}{<|`cWYXGO zY(Fvn0KT8(Z{8Vb!JlP4(E7$_#(B@yK)6(^}7uX9u?1b?uvQY@ZNd5 zIeEJ8@#MtmE0N#1 zA8a+7N50CEP`-IX?7;ZlpxqA-Lx4oPe{`?No?XB6E6;};C?A2lF&S}~pZ}SRbFdBE zAgz>?$cT$i#p@z?;0p=I5^|COF+DX!U$J$MB&z_)vb5!+N5xxs!V|~9j0PK~)PAPF zf<(aThwb5hjUuoO36>M)7X|wdD`#DmjI00rnY@p#?XiTo_+?NOOAox{f#t$&D=UXJ ztu`5o@>?(4-i)n;sICj&(s=qz+QO<;iT&;@@b zv#OV4fHc7e77G8;_p(*@kycNeOTf@s93f46Fjep$bhNFUsNQh*3#(01ma6K0>s{7t z@hCL(?Nnh;O3(V@L`RN^0@0awSWzniRIZetr3)>?0_- z;4?)?ZZ+8YP!PJ=h6q8w4%;21*E7AjtUCR3Dn6`seuG5Ufh_|ELCwH1>Bjv1(__XG2OXf#=Sjf5MRhud?#;@{pUE+1b>7ScJADLJ*V59{cM>#QgCTcp zLVt`O694B({jjmP%RN<}gS8>75}z#t;48PVu&_TYAY5=kRy4YPZt(sFBzSNbWUVy& z#QFDmUg8)%^Gnf~ob&pDc7A7~KiIPlw1lvR1YYe`6JtKmo}QU`QA4B1^UbSQw8Xo2 z?+&uba(vEyEhwJwqTR|;N$9Ru=$5$M!4W>!hnfaoi3=jpL$36z6l%vAm6bQ?_P>kh zwLJUNshhcPl;wo99C`7B*wMe29zCL86PK2b{Y8Ds@)4pTxlstgL^48D1o?jGf)kJR z4^=}$LuFF}rA@pS+sDVpV`5?qY%8=&y=#K^g~=^-BK@VZA@F^OPT1-6 zS9sQhU>6b8$t3(>EZX8soer{0zL4i+=W)vrHw4MX?a4tdo>(YI$WynZOZG5h|+&BjruQqY`(+5UXQ zVqeSd>V+hR=mZU_2Bam}DZ*Z6t6sG7Lro3IJ+X!7=kPGeRVL(Qe=!oZsA2IjaB{Ht zD`Dk@)cn=zjX0m91|uZ_2C8V~MxL!a)!p08H9K8q`qgGq&wLuepG}xfb-$4>iDc)J zb@FL!&(NiCPJR>;Qh^70+sRhic7)paViw+t*HIEe_43QTj-kS3&J)xGnAk~uOM!ub zAQpB_o1Ifr+eG$1$Z+i#%QOTxWgqSB?Xin2r4+$C`ntjPJ4`;;PmO0jYv~8bg4nrf zl#{-nH83i7n|lt&*5PQAN2+G(>WxsPB&{ts-Q6i=cmlA^!$ka&UgYwsQRS#a>^t{}PzXpkBd_GS^UcJ_2k=gfysF_t)4P zk60KL+a(tk7J_wqXPRa_cQ?Dt-+1{?pR$JMXS9!#ymV_yF9?s*{0sW-YHIGpdju?F zRIc9nZzV&+#ra~hNAbt5Dkqf% z$UFth%Nqd+m089iny#)cW`6ZApFh(&rD@jM|N0UOqISHjYj%10DLy_vS=sd{1Dl}G z(A`dznzh(TlVRm{B=UT%M- zMPJ{Q`)&+zQnd57J^ni~l9!ja%Nndp!y+3aM&oVGmjFL?O-(u3+3FWB3JDAEa&ny< zV7$s$;mQFcm4SLKr~qV9l@-fY9k@0USh`}WNTF$wN3{K6-p%VHhPP7YN} zzbp@H8i+N71qGQY2S6mpDtVcnj#zHQUPRF++tz$)wu%A(MyMXiyzwfv-~$?eMH(_c zcxA@Kuz)M{EF(SrwSfX&rZcRa0WtxsN2k;HmCo2ph>wQ*oKf`5z^2jVqkd+@|v&SsJhGz86 z4i0!L878d{G65JnTiv^Nj|h*5U>*%k3QdQzW_^?}>wLotVe zPyaBwOXF;e?T2KCUTY6u8(PvgA$ONuuVc#ZGSt$_!=SQ_%51s|jtU3}eBD%L{{ZGF z*p|u^%Zkf2?1+=`6iOvG7+5tW%B{o1d2y_j|7)KT`+eh_@8$0K)ut8}7N%DUVV?uq z^PTCvprK@$Wcljp0qCE4yh+->LN<~}g9N#2gZWYLM1GmgGZ+RZB>lAV{A z_xA1E@^Y_+rxIWULvecc+_`f}>-S{*VcS}qo<3&lyKayq+aiWpYx_gksB|Gx*Ekjy zRFDF`vtN<*+u-KoGlxy9M}^PaU`Ypl5-t>>*X7YJwtDGCe0p!EN>jY=AK+FQn!QV+I0mI5b+k8E9(` z)6FjD@wa@#=g%I$VsHL}_)T3-HTgwuZn#f15`oIYqo)8=VG3TEu>=%lg7k!Nbe3T! zV6Kj=`$fl-;YwYBV9hP~xTMCQv> zaCx%nu7pyl;0myoZRO??^4OXaeBJyHP4<;7!wa`%3Btn*YHCz|Q$0oN%Lz$I1;FaH zoUf4wi>)dF$Mwa80nxr0*s{ww9jUM&d3c{{lI^=URQ%zCN@WGb%vHa3J}^=1K7A?% z)P+YpSGMhtKvk*Ns#rz09JtNq`LQ-BCl+bApo#56SpsB<56Qt2?y+m){QPO`-j+M$D5vY2VlcRC* zBBdV+#IZ$>QhQ9DQ{pJXVA3fkX;-dYdc5R-d*W3g*K5bY~Ow|T9=j?Jsq8Iq~(Ny zQ;4KK8Ahg=pP#2x0`dtfC_TPuE=KP-|FaG%<8NbE1_uZGy?(3tE^tywj~%!@^f3hZ zNJk(N`?We-6BNeC;im)XyDEJvo#sb@TmZ#Rx=#0|C3;wa<`2wW=C$g8@WEP#B{$@Z z3Vq}kx0AiNQOos4}&4|SUmFjCVJ@K1x9 zvr%?tWrV#zDT${%bO>-Ni8H;Get*+%fOsO;s$HNg+GP@S7l)E^)S^Zn!OHs@&XjyM ztymd+NQ6g5-Y7~KSw&VMsxorkj&IBV!&rFrJ+)mzwHOP%c*z#1&RrD+HhnS#4Ff~R znA6q_AFVdX+W+XD)BPh8LOt>-xU5bUwo>u;PO`-3?$(h#ibRHZBQ=e!qBdxg5EL*W{<+|xH}d42|V-IjNoJ)~Eu zu7$#Jfp6!U-~oYSRaX3%n->fRZ7o9|JfNYYizrn3M7OXW5g$Lf!nFBq-~j!bdvTn{ z?dg1K!#j{MM=P2_6|UW7EInTE)|AEcO~m}X!teC0&+WY;`4lE5CVI*YUQ#4~Kc1HHtR-ZH z>a?{|rKuBZ6wexuj;?M;wzo3-1Gl*R))~Dc#=N++f`6e9wG*Cj&?MDFC^J2z+ux_w zBD8OJ#Zmf7)0e`%Ue$!}2hF&klO3&3`dpOf=mN=fWRq7Nl^F4=@I!247lrPYpRP2E zjEoEwZT`Q%65Frp^X6$@CC1SW*V^XOcTXMmagPtB2prUAqR4c4l9;H@DP_D(sg%T4 zBT3_a$HzB*OR_lxeV(aafT6FVF)N7eR{e{$icrlz!-GwcuLI23&1<*Q~`=Jl^% zKWwC3H}@wh*y+I($ZJbTC?`>?)U zJlK4`zI#K{lTujdQul;+0qA}XW7;SWJ>md6f{_RK_kMn`qPj(xOe)+?#@{^al&<&rs* zQ(Ec)H4Jgg@z+It*}$tY*a^?GeRu9KLS4ncz(6X-@CLFCOXf!uw%w7xfCd1*lQVqo zJt9O}Tgm>8iYi^bja)y~ZBpjaU>_OO+uMuOPL;BKWJL{@>5NC*g6 z!N(b>9C*cN`Y5e7ZioQ3bXXi$JK6WM*rsl*H4UK0cWuE^`7E|^Oj2A##L#!+f5IVe zdgK1vTsH@L8ri@VBn4n_>|fiT>+8+T%xsGX0{=K+eplxXcUvO7Nxk#CCKq@Euf~T8 zq3Py{oF6S>M9aXyDR%u65RZ{|@ha!^_knj* z#LHK&g3;S>awUHOoe95syx+F%jIr6%@+~R9ZT;LccH9C-w5N4;b^H03-cSu06qY%8 zOhBaB!k?NsGFELwV)~utcXuwP=c?i&^g(-WrHGm<3SZj$g}$HI<*!%c(z#!tz{Prb z@aJ~Q*f81W_q4v1&vetg8>O41C8|vt9UEu3S}erHV!wQUPezcLQqd9A{{T`I6&5lz z;|-oZeR@;wASZC{0wAPH0fq=w(W<8uG5fumLx^B`a9Kbba05=WHP_~r^}I-BlqB5} zB_kovu2)S?RHnatnX8|rvJhl@3ZgKf(1Rz;A63?p6YmspO@|M9Y(q0#^faiFwm8TM z8u~CAvLD-OH*elFd1@dO<-4_dKTX-wvy3xrN7wB_d3i~RxCW_y%CFyZ{qfWJjw9+> zIu}#`psK5@Rest@<>luGJ@D30QP`LbxIld$)oaK>>6X+C)P}!DnzmO*p4s-7zt+z} zj}eP^3w;v@2M5&<)?uHCrKKg@fWdT0MdbtV%eb8s6tT2)bhoUmoMem?6|3ONO1jfU z=SoI<6gH8_NchtxrKi`~6frk`>APC&O9lAgT=wpdSs5CY1BEwDX-mfz=_I z=Kcrke$Qk5tj4CMO84I{0AC)~F!NnJtD*7z&8_J5u$=_C9YL&|gJJgi>6DI%u9K2+ zSv{5d4Gj&A2`u!13ky!Rk$9hx=Tq0Sy7rvmM&l1luPg6{$gYPEA6{udL*#Xg?P~=I zi8raKHL2Tp{lA+A5xj=Oz<=|AEEQWY^=zj~h?2V=L3}VSL^!h4)xmD#Ok6um*O@TZ0cpQ?$mg&A^rz zGR43CYK2)j8pK*!TIJ49qN81L*1>f;P)ILVF+5f6A>nxR#HtiG zT!}+ReE)PKX2%>%ss}H|9J^!Wu{_PL!2lSMsqf&BSAYIdgL!>J!wp>?Ztng{-x;q+ zgX(J;e*yl9G-T-r3Su8XgFHqq#&HhC4{DJ-pWU4Dlg3;pPDB`0_-x8F3AyRye#F?X_|;Is$fYJtn>hi(Ga#R;M784pm{9 zj+yysU%FI1u5#wgn3GJHGwP62)QX0FXH>*uGqbW9ijInli=REr#2p+K=33cZOJpP` z-|>`5*GlhYvl3?&36^pE^{8HZS_{jkr^vQ1j#CP^Gpqm%lsyw57O2(tkXM;HZ>-qs zK&M+Z+WcJZn>27~kLP7W!^x~?P-w1~HyemV1M`PHU}a-Nu-PWK{iUb(De>}))KuXs zc^@kLahCMjY=_`8InQqf3bHPfEV3?{xVuL;`}#dB%?o}5)aw9n)vIgFsjB60Fr>;*n&F|7T zFu#QQlou~NGI*lv(Vo3} zj3ra&mbbSzBEd6ZWyLMb8p|{wAV3x$85znyjuCdyBUUA)w2tW|XJuKVn<8g9c$~lh zhHP)V)Z;Eo{EP`fD*t@?bXMm#LZ4$o>RFjn^=<|i zSqWlP_}^)IeRy}p;PJX|;asOz46JciN|?fysRZr=kOWknot@3iJtlJWJvt6Kh9xaM z)lT(RZyeIO4Vkoye=vp6vJH?anjm=ml!EHr5^+nEuV(=+;LvlDz)=gP|DN)I@%@|Y zUM6Q#TauCv<#%0FVP|LWeXQs@H8qv_LzwOqv9X+6$4*f5#$NnLD{5ShjbhX3K&n<9 zEx^P)i_xAv7kIlli=X?VtXtNpy)6kLX>g=_BCZegr6=1@(5v+JY?f1L05;*Hg7N@8 zzjlrH7B)8KRMQoAM823iYHLRp{W9DyxK;G0=LrBYogcAD`dQ`*m)zW>qIKzsjotq~ z2-0M;dxeFPw$8=k>Nsp0PqXeeEb3?a18vk$cJqbN{3dy`dzIT;J@?BTOL8#vUHQtc z*mw+YgxRuD5|yOwjABo;#sIQT|F{@zG{$Ox+0eTM_TklH__Eb60+T3 zD2l?#A2hx-pYePoDG^aoSDzH1YaBXCT6gbdXZId!;pGfOVylk2hyG#Vjr#o}49cjh z5QZaPVsSr6Z3x>))+#zZdI%;upnIHF^$Hc}TNk-<$~cXp*wFCyrMscT1X@1)^x2zeXR_orxCLbr^6 zBcsI|O!ZgvlbRL_J-k4g&5C=_8Ke+Fk>=sNs%{f-j=3g3EO_{H6L(_Lx{dm6plP3#l`I|r~oWrPa2fGy{O~k5%-+wQE`EnGM$#nhFZy;~9e?~_wxE-VpvWDQU!^gsH z_+=Xlts21KbUSy>mG}~)4BR`IR>=LHp#_R`%Xmpki^=u&SEF0EGK_O(CMU0R?xVe@ z!fl`P4T%M)M_0Gj gpu) |> cpu # first row is prob. of true, second row p(fa mean((out2[1,:] .> 0.5) .== truth) # accuracy 94% so far! ``` -![](../assets/oneminute.png) +![](../assets/quickstart/oneminute.png) ```julia using Plots # to draw the above figure @@ -63,12 +63,18 @@ p_done = scatter(noisy[1,:], noisy[2,:], zcolor=out2[1,:], title="Trained networ plot(p_true, p_raw, p_done, layout=(1,3), size=(1000,330)) ``` +```@raw html + +``` + Here's the loss during training: ```julia -plot(losses; xaxis=(:log10, "iteration"), yaxis="loss", label="per batch") +plot(losses; xaxis=(:log10, "iteration"), + yaxis="loss", label="per batch") n = length(loader) -plot!(n:n:length(losses), mean.(Iterators.partition(losses, n)), label="epoch mean") +plot!(n:n:length(losses), mean.(Iterators.partition(losses, n)), + label="epoch mean", dpi=200) ``` This XOR ("exclusive or") problem is a variant of the famous one which drove Minsky and Papert to invent deep neural networks in 1969. For small values of "deep" -- this has one hidden layer, while earlier perceptrons had none. (What they call a hidden layer, Flux calls the output of the first layer, `model[1](noisy)`.) From c9cde50999ae66297ef56375c1b59e420584bbac Mon Sep 17 00:00:00 2001 From: Michael Abbott <32575566+mcabbott@users.noreply.github.com> Date: Sat, 26 Nov 2022 15:53:12 -0500 Subject: [PATCH 15/15] one more .. perhaps --- docs/src/models/quickstart.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/src/models/quickstart.md b/docs/src/models/quickstart.md index ea0c972ed1..f93be4de95 100644 --- a/docs/src/models/quickstart.md +++ b/docs/src/models/quickstart.md @@ -64,7 +64,7 @@ plot(p_true, p_raw, p_done, layout=(1,3), size=(1000,330)) ``` ```@raw html - + ``` Here's the loss during training: