Merge pull request #218 from JuliaAI/dev

For a 0.8.6 release

Author: ablaom

Project.toml

@@ -1,7 +1,7 @@
 name = "MLJTuning"
 uuid = "03970b2e-30c4-11ea-3135-d1576263f10f"
 authors = ["Anthony D. Blaom <anthony.blaom@gmail.com>"]
-version = "0.8.5"
+version = "0.8.6"
 
 [deps]
 ComputationalResources = "ed09eef8-17a6-5b46-8889-db040fac31e3"

README.md

@@ -1,10 +1,10 @@
 # MLJTuning
 
 Hyperparameter optimization for
-[MLJ](https://github.com/alan-turing-institute/MLJ.jl) machine
+[MLJ](https://github.com/JuliaAI/MLJ.jl) machine
 learning models.
 
-See [**Tuning Models · MLJ**](https://alan-turing-institute.github.io/MLJ.jl/dev/tuning_models) for usage examples.
+See [**Tuning Models · MLJ**](https://JuliaAI.github.io/MLJ.jl/dev/tuning_models) for usage examples.
 
 [![Build Status](https://github.com/JuliaAI/MLJTuning.jl/workflows/CI/badge.svg)](https://github.com/JuliaAI/MLJTuning.jl/actions)
 [![codecov.io](http://codecov.io/github/JuliaAI/MLJTuning.jl/coverage.svg?branch=master)](http://codecov.io/github/JuliaAI/MLJTuning.jl?branch=master)
@@ -17,17 +17,17 @@ See [**Tuning Models · MLJ**](https://alan-turing-institute.github.io/MLJ.jl/de
  - [How do I implement a new selection heuristic?](#how-do-i-implement-a-new-selection-heuristic)
 
 *Note:* This component of the [MLJ
-  stack](https://github.com/alan-turing-institute/MLJ.jl#the-mlj-universe)
+  stack](https://github.com/JuliaAI/MLJ.jl#the-mlj-universe)
   applies to MLJ versions 0.8.0 and higher. Prior to 0.8.0, tuning
   algorithms resided in
-  [MLJ](https://github.com/alan-turing-institute/MLJ.jl).
+  [MLJ](https://github.com/JuliaAI/MLJ.jl).
 
 
 ## Who is this repo for?
 
 This repository is not intended to be directly imported by the general
 MLJ user. Rather, MLJTuning is a dependency of the
-[MLJ](https://github.com/alan-turing-institute/MLJ.jl) machine
+[MLJ](https://github.com/JuliaAI/MLJ.jl) machine
 learning platform, which allows MLJ users to perform a variety of
 hyperparameter optimization tasks from there.
 
@@ -38,9 +38,9 @@ importing MLJTuning into a third-party package and implementing
 MLJTuning's [tuning strategy interface](#how-do-i-implement-a-new-tuning-strategy).
 
 MLJTuning is a component of the [MLJ
-  stack](https://github.com/alan-turing-institute/MLJ.jl#the-mlj-universe)
+  stack](https://github.com/JuliaAI/MLJ.jl#the-mlj-universe)
   which does not have
-  [MLJModels](https://github.com/alan-turing-institute/MLJModels.jl)
+  [MLJModels](https://github.com/JuliaAI/MLJModels.jl)
   as a dependency (no ability to search and load registered MLJ
   models). It does however depend on
   [MLJBase](https://github.com/JuliaAI/MLJBase.jl) and,
@@ -94,7 +94,7 @@ This repository contains:
 
 - a selection of **implementations** of the tuning strategy interface,
   currently all those accessible from
-  [MLJ](https://github.com/alan-turing-institute/MLJ.jl) itself.
+  [MLJ](https://github.com/JuliaAI/MLJ.jl) itself.
 
 - the code defining the MLJ functions `learning_curves!` and `learning_curve` as
   these are essentially one-dimensional grid searches
@@ -103,12 +103,12 @@ This repository contains:
 ## How do I implement a new tuning strategy?
 
 This document assumes familiarity with the [Evaluating Model
-Performance](https://alan-turing-institute.github.io/MLJ.jl/dev/evaluating_model_performance/)
+Performance](https://JuliaAI.github.io/MLJ.jl/dev/evaluating_model_performance/)
 and [Performance
-Measures](https://alan-turing-institute.github.io/MLJ.jl/dev/performance_measures/)
+Measures](https://JuliaAI.github.io/MLJ.jl/dev/performance_measures/)
 sections of the MLJ manual. Tuning itself, from the user's
 perspective, is described in [Tuning
-Models](https://alan-turing-institute.github.io/MLJ.jl/dev/tuning_models/).
+Models](https://JuliaAI.github.io/MLJ.jl/dev/tuning_models/).
 
 
 ### Overview
@@ -158,11 +158,11 @@ begin, on the basis of the specific strategy and a user-specified
   measures that do not report per-observation values
   (`reports_per_observation(measure) = false`) such as `auc`. See
   [Evaluating Model
-  Performance](https://alan-turing-institute.github.io/MLJ.jl/dev/evaluating_model_performance/)
+  Performance](https://JuliaAI.github.io/MLJ.jl/dev/evaluating_model_performance/)
   for details. There is a trait for measures called `orientation`
   which is `:loss` for measures you ordinarily want to minimize, and
   `:score` for those you want to maximize. See [Performance
-  measures](https://alan-turing-institute.github.io/MLJ.jl/dev/performance_measures/)
+  measures](https://JuliaAI.github.io/MLJ.jl/dev/performance_measures/)
   for further details.
 
 - A *tuning strategy* is an instance of some subtype `S <:
@@ -233,7 +233,7 @@ wrapper. A model is tuned by *fitting* the wrapped model to data
 process determines the optimal model, as defined by the selection
 heuristic (see above). To use the optimal model one *predicts* using
 the wrapped model. For more detail, see the [Tuning
-Models](https://alan-turing-institute.github.io/MLJ.jl/dev/tuning_models/)
+Models](https://JuliaAI.github.io/MLJ.jl/dev/tuning_models/)
 section of the MLJ manual.
 
 In setting up a tuning task, the user constructs an instance of the
@@ -371,7 +371,7 @@ composite models this might be a be an `Expr`, such as
 Use the `iterator` and `sampler` methods to convert ranges into
 one-dimensional grids or for random sampling, respectively. See the
 [tuning
-section](https://alan-turing-institute.github.io/MLJ.jl/dev/tuning_models/#API-1)
+section](https://JuliaAI.github.io/MLJ.jl/dev/tuning_models/#API-1)
 of the MLJ manual or doc-strings for more on these methods and the
 `Grid` and `RandomSearch` implementations.
 
@@ -481,7 +481,7 @@ If more models are returned than needed (because including them would
 create a history whose length exceeds the user-specified number of
 iterations `tuned_model.n`) then the surplus models are saved, for use
 in a ["warm
-restart"](https://alan-turing-institute.github.io/MLJ.jl/dev/machines/#Warm-restarts)
+restart"](https://JuliaAI.github.io/MLJ.jl/dev/machines/#Warm-restarts)
 of tuning, when the user increases `tuned_model.n`. The remaining
 models are then evaluated and these evaluations are added to the
 history. **In any warm restart, no new call to `models` will be made

src/tuned_models.jl

@@ -33,7 +33,7 @@ warn_double_spec(arg, model) =
 const ProbabilisticTypes = Union{Probabilistic, MLJBase.MLJModelInterface.ProbabilisticDetector}
 const DeterministicTypes = Union{Deterministic, MLJBase.MLJModelInterface.DeterministicDetector}
 
-mutable struct DeterministicTunedModel{T,M<:DeterministicTypes} <: MLJBase.Deterministic
+mutable struct DeterministicTunedModel{T,M<:DeterministicTypes,L} <: MLJBase.Deterministic
     model::M
     tuning::T  # tuning strategy
     resampling # resampling strategy
@@ -51,9 +51,10 @@ mutable struct DeterministicTunedModel{T,M<:DeterministicTypes} <: MLJBase.Deter
     check_measure::Bool
     cache::Bool
     compact_history::Bool
+    logger::L
 end
 
-mutable struct ProbabilisticTunedModel{T,M<:ProbabilisticTypes} <: MLJBase.Probabilistic
+mutable struct ProbabilisticTunedModel{T,M<:ProbabilisticTypes,L} <: MLJBase.Probabilistic
     model::M
     tuning::T  # tuning strategy
     resampling # resampling strategy
@@ -71,10 +72,11 @@ mutable struct ProbabilisticTunedModel{T,M<:ProbabilisticTypes} <: MLJBase.Proba
     check_measure::Bool
     cache::Bool
     compact_history::Bool
+    logger::L
 end
 
-const EitherTunedModel{T,M} =
-    Union{DeterministicTunedModel{T,M},ProbabilisticTunedModel{T,M}}
+const EitherTunedModel{T,M,L} =
+    Union{DeterministicTunedModel{T,M,L},ProbabilisticTunedModel{T,M,L}}
 
 MLJBase.caches_data_by_default(::Type{<:EitherTunedModel}) = false
 
@@ -279,6 +281,7 @@ function TunedModel(
     check_measure=true,
     cache=true,
     compact_history=true,
+    logger=nothing
     )
 
     # user can specify model as argument instead of kwarg:
@@ -342,6 +345,9 @@ function TunedModel(
     # get the tuning type parameter:
     T = typeof(tuning)
 
+    # get the logger type parameter:
+    L = typeof(logger)
+
     args = (
         model,
         tuning,
@@ -360,12 +366,13 @@ function TunedModel(
         check_measure,
         cache,
         compact_history,
+        logger
     )
 
     if M <: DeterministicTypes
-        tuned_model = DeterministicTunedModel{T,M}(args...)
+        tuned_model = DeterministicTunedModel{T,M,L}(args...)
     elseif M <: ProbabilisticTypes
-        tuned_model = ProbabilisticTunedModel{T,M}(args...)
+        tuned_model = ProbabilisticTunedModel{T,M,L}(args...)
     else
         throw(ERR_MODEL_TYPE)
     end
@@ -591,7 +598,7 @@ function assemble_events!(metamodels,
             end
         end
         # One resampling_machine per task
-         machs = [resampling_machine,
+        machs = [resampling_machine,
                  [machine(Resampler(
                      model= resampling_machine.model.model,
                      resampling    = resampling_machine.model.resampling,
@@ -603,9 +610,9 @@ function assemble_events!(metamodels,
                      repeats       = resampling_machine.model.repeats,
                      acceleration  = resampling_machine.model.acceleration,
                      cache         = resampling_machine.model.cache,
-                     compact       = resampling_machine.model.compact
-                 ), resampling_machine.args...; cache=false) for
-                     _ in 2:length(partitions)]...]
+                     compact       = resampling_machine.model.compact,
+                     logger        = resampling_machine.model.logger),
+                resampling_machine.args...; cache=false) for _ in 2:length(partitions)]...]
 
         @sync for (i, parts) in enumerate(partitions)
             Threads.@spawn begin
@@ -740,8 +747,8 @@ function finalize(tuned_model,
     return fitresult, meta_state, report
 end
 
-function MLJBase.fit(tuned_model::EitherTunedModel{T,M},
-                     verbosity::Integer, data...) where {T,M}
+function MLJBase.fit(tuned_model::EitherTunedModel{T,M,L},
+                     verbosity::Integer, data...) where {T,M,L}
     tuning = tuned_model.tuning
     model = tuned_model.model
     _range = tuned_model.range
@@ -769,6 +776,7 @@ function MLJBase.fit(tuned_model::EitherTunedModel{T,M},
         acceleration  = tuned_model.acceleration_resampling,
         cache         = tuned_model.cache,
         compact       = tuned_model.compact_history,
+        logger        = tuned_model.logger
     )
     resampling_machine = machine(resampler, data...; cache=false)
     history, state = build!(nothing, n, tuning, model, model_buffer, state,
@@ -900,9 +908,9 @@ end
 ## METADATA
 
 MLJBase.is_wrapper(::Type{<:EitherTunedModel}) = true
-MLJBase.supports_weights(::Type{<:EitherTunedModel{<:Any,M}}) where M =
+MLJBase.supports_weights(::Type{<:EitherTunedModel{<:Any,M,L}}) where {M,L} =
     MLJBase.supports_weights(M)
-MLJBase.supports_class_weights(::Type{<:EitherTunedModel{<:Any,M}}) where M =
+MLJBase.supports_class_weights(::Type{<:EitherTunedModel{<:Any,M,L}}) where {M,L} =
     MLJBase.supports_class_weights(M)
 MLJBase.load_path(::Type{<:ProbabilisticTunedModel}) =
     "MLJTuning.ProbabilisticTunedModel"
@@ -914,9 +922,9 @@ MLJBase.package_uuid(::Type{<:EitherTunedModel}) =
 MLJBase.package_url(::Type{<:EitherTunedModel}) =
     "https://github.com/alan-turing-institute/MLJTuning.jl"
 MLJBase.package_license(::Type{<:EitherTunedModel}) = "MIT"
-MLJBase.is_pure_julia(::Type{<:EitherTunedModel{T,M}}) where {T,M} =
+MLJBase.is_pure_julia(::Type{<:EitherTunedModel{T,M,L}}) where {T,M,L} =
     MLJBase.is_pure_julia(M)
-MLJBase.input_scitype(::Type{<:EitherTunedModel{T,M}}) where {T,M} =
+MLJBase.input_scitype(::Type{<:EitherTunedModel{T,M,L}}) where {T,M,L} =
     MLJBase.input_scitype(M)
-MLJBase.target_scitype(::Type{<:EitherTunedModel{T,M}}) where {T,M} =
+MLJBase.target_scitype(::Type{<:EitherTunedModel{T,M,L}}) where {T,M,L} =
     MLJBase.target_scitype(M)