Release 1.2.0 stable

XGBoost4J-Spark now supports the GPU algorithm (#5171)

• Now XGBoost4J-Spark is able to leverage NVIDIA GPU hardware to speed up training.
• There is on-going work for accelerating the rest of the data pipeline with NVIDIA GPUs (#5950, #5972).

XGBoost now supports CUDA 11 (#5808)

• It is now possible to build XGBoost with CUDA 11. Note that we do not yet distribute pre-built binaries built with CUDA 11; all current distributions use CUDA 10.0.

Better guidance for persisting XGBoost models in an R environment (#5940, #5964)

• Users are strongly encouraged to use xgb.save() and xgb.save.raw() instead of saveRDS(). This is so that the persisted models can be accessed with future releases of XGBoost.
• The previous release (1.1.0) had problems loading models that were saved with saveRDS(). This release adds a compatibility layer to restore access to the old RDS files. Note that this is meant to be a temporary measure; users are advised to stop using saveRDS() and migrate to xgb.save() and xgb.save.raw().

New objectives and metrics

• The pseudo-Huber loss reg:pseudohubererror is added (#5647). The corresponding metric is mphe. Right now, the slope is hard-coded to 1.
• The Accelerated Failure Time objective for survival analysis (survival:aft) is now accelerated on GPUs (#5714, #5716). The survival metrics aft-nloglik and interval-regression-accuracy are also accelerated on GPUs.

Improved integration with scikit-learn

• Added n_features_in_ attribute to the scikit-learn interface to store the number of features used (#5780). This is useful for integrating with some scikit-learn features such as StackingClassifier. See this link for more details.
• XGBoostError now inherits ValueError, which conforms scikit-learn's exception requirement (#5696).

Improved integration with Dask

• The XGBoost Dask API now exposes an asynchronous interface (#5862). See the document for details.
• Zero-copy ingestion of GPU arrays via DaskDeviceQuantileDMatrix (#5623, #5799, #5800, #5803, #5837, #5874, #5901): Previously, the Dask interface had to make 2 data copies: one for concatenating the Dask partition/block into a single block and another for internal representation. To save memory, we introduce DaskDeviceQuantileDMatrix. As long as Dask partitions are resident in the GPU memory, DaskDeviceQuantileDMatrix is able to ingest them directly without making copies. This matrix type wraps DeviceQuantileDMatrix.
• The prediction function now returns GPU Series type if the input is from Dask-cuDF (#5710). This is to preserve the input data type.

Robust handling of external data types (#5689, #5893)

• As we support more and more external data types, the handling logic has proliferated all over the code base and became hard to keep track. It also became unclear how missing values and threads are handled. We refactored the Python package code to collect all data handling logic to a central location, and now we have an explicit list of of all supported data types.

Improvements in GPU-side data matrix (DeviceQuantileDMatrix)

• The GPU-side data matrix now implements its own quantile sketching logic, so that data don't have to be transported back to the main memory (#5700, #5747, #5760, #5846, #5870, #5898). The GK sketching algorithm is also now better documented.
  • Now we can load extremely sparse dataset like URL, although performance is still sub-optimal.
• The GPU-side data matrix now exposes an iterative interface (#5783), so that users are able to construct a matrix from a data iterator. See the Python demo.

New language binding: Swift (#5728)

• Visit https://github.com/kongzii/SwiftXGBoost for more details.

Robust model serialization with JSON (#5772, #5804, #5831, #5857, #5934)

• We continue efforts from the 1.0.0 release to adopt JSON as the format to save and load models robustly.
• JSON model IO is significantly faster and produces smaller model files.
• Round-trip reproducibility is guaranteed, via the introduction of an efficient float-to-string conversion algorithm known as the Ryū algorithm. The conversion is locale-independent, producing consistent numeric representation regardless of the locale setting of the user's machine.
• We fixed an issue in loading large JSON files to memory.
• It is now possible to load a JSON file from a remote source such as S3.

Performance improvements

• CPU hist tree method optimization
  • Skip missing lookup in hist row partitioning if data is dense. (#5644)
  • Specialize training procedures for CPU hist tree method on distributed environment. (#5557)
  • Add single point histogram for CPU hist. Previously gradient histogram for CPU hist is hard coded to be 64 bit, now users can specify the parameter single_precision_histogram to use 32 bit histogram instead for faster training performance. (#5624, #5811)
• GPU hist tree method optimization
  • Removed some unnecessary synchronizations and better memory allocation pattern. (#5707)
  • Optimize GPU Hist for wide dataset. Previously for wide dataset the atomic operation is performed on global memory, now it can run on shared memory for faster histogram building. But there's a known small regression on GeForce cards with dense data. (#5795, #5926, #5948, #5631)

API additions

• Support passing fmap to importance plot (#5719). Now importance plot can show actual names of features instead of default ones.
• Support 64bit seed. (#5643)
• A new C API XGBoosterGetNumFeature is added for getting number of features in booster (#5856).
• Feature names and feature types are now stored in C++ core and saved in binary DMatrix (#5858).

Breaking: The predict() method of DaskXGBClassifier now produces class predictions (#5986). Use predict_proba() to obtain probability predictions.

• Previously, DaskXGBClassifier.predict() produced probability predictions. This is inconsistent with the behavior of other scikit-learn classifiers, where predict() returns class predictions. We make a breaking change in 1.2.0 release so that DaskXGBClassifier.predict() now correctly produces class predictions and thus behave like other scikit-learn classifiers. Furthermore, we introduce the predict_proba() method for obtaining probability predictions, again to be in line with other scikit-learn classifiers.

Breaking: Custom evaluation metric now receives raw prediction (#5954)

• Previously, the custom evaluation metric received a transformed prediction result when used with a classifier. Now the custom metric will receive a raw (untransformed) prediction and will need to transform the prediction itself. See demo/guide-python/custom_softmax.py for an example.
• This change is to make the custom metric behave consistently with the custom objective, which already receives raw prediction (#5564).

Breaking: XGBoost4J-Spark now requires Spark 3.0 and Scala 2.12 (#5836, #5890)

• Starting with version 3.0, Spark can manage GPU resources and allocate them among executors.
• Spark 3.0 dropped support for Scala 2.11 and now only supports Scala 2.12. Thus, XGBoost4J-Spark also only supports Scala 2.12.

Breaking: XGBoost Python package now requires Python 3.6 and later (#5715)

• Python 3.6 has many useful features such as f-strings.

Breaking: XGBoost now adopts the C++14 standard (#5664)

• Make sure to use a sufficiently modern C++ compiler that supports C++14, such as Visual Studio 2017, GCC 5.0+, and Clang 3.4+.

Bug-fixes

• Fix a data race in the prediction function (#5853). As a byproduct, the prediction function now uses a thread-local data store and became thread-safe.
• Restore capability to run prediction when the test input has fewer features than the training data (#5955). This capability is necessary to support predicting with LIBSVM inputs. The previous release (1.1) had broken this capability, so we restore it in this version with better tests.
• Fix OpenMP build with CMake for R package, to support CMake 3.13 (#5895).
• Fix Windows 2016 build (#5902, #5918).
• Fix edge cases in scikit-learn interface with Pandas input by disabling feature validation. (#5953)
• [R] Enable weighted learning to rank (#5945)
• [R] Fix early stopping with custom objective (#5923)
• Fix NDK Build (#5886)
• Add missing explicit template specializations for greater portability (#5921)
• Handle empty rows in data iterators correctly (#5929). This bug affects file loader and JVM data frames.
• Fix IsDense (#5702)
• [jvm-packages] Fix wrong method name setAllowZeroForMissingValue (#5740)
• Fix shape inference for Dask predict (#5989)

Usability Improvements, Documentation

• [Doc] Document that CUDA 10.0 is required (#5872)
• Refactored command line interface (CLI). Now CLI is able to handle user errors and output basic document. (#5574)
• Better error handling in Python: use raise from syntax to preserve full stacktrace (#5787).
• The JSON model dump now has a formal schema (#5660, #5818). The benefit is to prevent dump_model() function from breaking. See this document to understand the difference between saving and dumping models.
• Add a reference to the GPU external memory paper (#5684)
• Document more objective parameters in the R package (#5682)
• Document the existence of pre-built binary wheels for MacOS (#5711)
• Remove `m...

Release Candidate 2 of version 1.2.0

#5970

R package: xgboost_1.2.0.1.tar.gz (Manual: xgboost_1.2.0.1-manual.pdf)

Release Candidate of version 1.2.0

#5970

R package: xgboost_1.2.0.1.tar.gz (Manual: xgboost_1.2.0.1-manual.pdf)

1.1.1 Patch Release

This patch release applies the following patches to 1.1.0 release:

• CPU performance improvement in the PyPI wheels (#5720)
• Fix loading old model. (#5724)
• Install pkg-config file (#5744)

Release 1.1.0 stable

Better performance on multi-core CPUs (#5244, #5334, #5522)

• Poor performance scaling of the hist algorithm for multi-core CPUs has been under investigation (#3810). #5244 concludes the ongoing effort to improve performance scaling on multi-CPUs, in particular Intel CPUs. Roadmap: #5104
• #5334 makes steps toward reducing memory consumption for the hist tree method on CPU.
• #5522 optimizes random number generation for data sampling.

Deterministic GPU algorithm for regression and classification (#5361)

• GPU algorithm for regression and classification tasks is now deterministic.
• Roadmap: #5023. Currently only single-GPU training is deterministic. Distributed training with multiple GPUs is not yet deterministic.

Improve external memory support on GPUs (#5093, #5365)

• Starting from 1.0.0 release, we added support for external memory on GPUs to enable training with larger datasets. Gradient-based sampling (#5093) speeds up the external memory algorithm by intelligently sampling a subset of the training data to copy into the GPU memory. Learn more about out-of-core GPU gradient boosting.
• GPU-side data sketching now works with data from external memory (#5365).

Parameter validation: detection of unused or incorrect parameters (#5477, #5569, #5508)

• Mis-spelled training parameter is a common user mistake. In previous versions of XGBoost, mis-spelled parameters were silently ignored. Starting with 1.0.0 release, XGBoost will produce a warning message if there is any unused training parameters. The 1.1.0 release makes parameter validation available to the scikit-learn interface (#5477) and the R binding (#5569).

Thread-safe, in-place prediction method (#5389, #5512)

• Previously, the prediction method was not thread-safe (#5339). This release adds a new API function inplace_predict() that is thread-safe. It is now possible to serve concurrent requests for prediction using a shared model object.
• It is now possible to compute prediction in-place for selected data formats (numpy.ndarray / scipy.sparse.csr_matrix / cupy.ndarray / cudf.DataFrame / pd.DataFrame) without creating a DMatrix object.

Addition of Accelerated Failure Time objective for survival analysis (#4763, #5473, #5486, #5552, #5553)

• Survival analysis (regression) models the time it takes for an event of interest to occur. The target label is potentially censored, i.e. the label is a range rather than a single number. We added a new objective survival:aft to support survival analysis. Also added is the new API to specify the ranged labels. Check out the tutorial and the demos.
• GPU support is work in progress (#5714).

Improved installation experience on Mac OSX (#5597, #5602, #5606, #5701)

• It only takes two commands to install the XGBoost Python package: brew install libomp followed by pip install xgboost. The installed XGBoost will use all CPU cores. Even better, starting with this release, we distribute pre-compiled binary wheels targeting Mac OSX. Now the install command pip install xgboost finishes instantly, as it no longer compiles the C++ source of XGBoost. The last three Mac versions (High Sierra, Mojave, Catalina) are supported.
• R package: the 1.1.0 release fixes the error Initializing libomp.dylib, but found libomp.dylib already initialized (#5701)

Ranking metrics are now accelerated on GPUs (#5380, #5387, #5398)

GPU-side data matrix to ingest data directly from other GPU libraries (#5420, #5465)

• Previously, data on GPU memory had to be copied back to the main memory before it could be used by XGBoost. Starting with 1.1.0 release, XGBoost provides a dedicated interface (DeviceQuantileDMatrix) so that it can ingest data from GPU memory directly. The result is that XGBoost interoperates better with GPU-accelerated data science libraries, such as cuDF, cuPy, and PyTorch.
• Set device in device dmatrix. (#5596)

Robust model serialization with JSON (#5123, #5217)

• We continue efforts from the 1.0.0 release to adopt JSON as the format to save and load models robustly. Refer to the release note for 1.0.0 to learn more.
• It is now possible to store internal configuration of the trained model (Booster) object in R as a JSON string (#5123, #5217).

Improved integration with Dask

• Pass through verbose parameter for dask fit (#5413)
• Use DMLC_TASK_ID. (#5415)
• Order the prediction result. (#5416)
• Honor nthreads from dask worker. (#5414)
• Enable grid searching with scikit-learn. (#5417)
• Check non-equal when setting threads. (#5421)
• Accept other inputs for prediction. (#5428)
• Fix missing value for scikit-learn interface. (#5435)

XGBoost4J-Spark: Check number of columns in the data iterator (#5202, #5303)

• Before, the native layer in XGBoost did not know the number of columns (features) ahead of time and had to guess the number of columns by counting the feature index when ingesting data. This method has a failure more in distributed setting: if the training data is highly sparse, some features may be completely missing in one or more worker partitions. Thus, one or more workers may deduce an incorrect data shape, leading to crashes or silently wrong models.
• Enforce correct data shape by passing the number of columns explicitly from the JVM layer into the native layer.

Major refactoring of the DMatrix class

• Continued from 1.0.0 release.
• Remove update prediction cache from predictors. (#5312)
• Predict on Ellpack. (#5327)
• Partial rewrite EllpackPage (#5352)
• Use ellpack for prediction only when sparsepage doesn't exist. (#5504)
• RFC: #4354, Roadmap: #5143

Breaking: XGBoost Python package now requires Pip 19.0 and higher (#5589)

• Your Linux machine may have an old version of Pip and may attempt to install a source package, leading to long installation time. This is because we are now using manylinux2010 tag in the binary wheel release. Ensure you have Pip 19.0 or newer by running python3 -m pip -V to check the version. Upgrade Pip with command

python3 -m pip install --upgrade pip

Upgrading to latest pip allows us to depend on newer versions of system libraries. TensorFlow also requires Pip 19.0+.

Breaking: GPU algorithm now requires CUDA 10.0 and higher (#5649)

• CUDA 10.0 is necessary to make the GPU algorithm deterministic (#5361).

Breaking: silent parameter is now removed (#5476)

• Please use verbosity instead.

Breaking: Set output_margin to True for custom objectives (#5564)

• Now both R and Python interface custom objectives get un-transformed (raw) prediction outputs.

Breaking: Makefile is now removed. We use CMake exclusively to build XGBoost (#5513)

• Exception: the R package uses Autotools, as the CRAN ecosystem did not yet adopt CMake widely.

Breaking: distcol updater is now removed (#5507)

• The distcol updater has been long broken, and currently we lack resources to implement a working implementation from scratch.

Deprecation notices

• Python 3.5. This release is the last release to support Python 3.5. The following release (1.2.0) will require Python 3.6.
• Scala 2.11. Currently XGBoost4J supports Scala 2.11. However, if a future release of XGBoost adopts Spark 3, it will not support Scala 2.11, as Spark 3 requires Scala 2.12+. We do not yet know which XGBoost release will adopt Spark 3.

Known limitations

• (Python package) When early stopping is activated with early_stopping_rounds at training time, the prediction method (xgb.predict()) behaves in a surprising way. If XGBoost runs for M rounds and chooses iteration N (N < M) as the best iteration, then the prediction method will use M trees by default. To use the best iteration (N trees), users will need to manually take the best iteration field bst.best_iteration and pass it as the ntree_limit argument to xgb.predict(). See #5209 and #4052 for additional context.
• GPU ranking objective is currently not deterministic (#5561).
• When training parameter reg_lambda is set to zero, some leaf nodes may be assigned a NaN value. (See discussion.) For now, please set reg_lambda to a nonzero value.

Community and Governance

• The XGBoost Project Management Committee (PMC) is pleased to announce a new committer: Egor Smirnov (@SmirnovEgorRu). He has led a major initiative to improve the performance of XGBoost on multi-core CPUs.

Bug-fixes

• Improved compatibility with scikit-learn (#5255, #5505, #5538)
• Remove f-string, since it's not supported by Python 3.5 (#5330). Note that Python 3.5 support is deprecated and schedule to be dropped in the upcoming release (1.2.0).
• Fix the pruner so that it doesn't prune the same branch twice (#5335)
• Enforce only major version in JSON model schema (#5336). Any major revision of the model schema would bump up the major version.
• Fix a small typo in sklearn.py that broke multiple eval metrics (#5341)
• Restore loading model from a memory buffer (#5360)
• Define lazy isinstance for Python compat (#5364)
• [R] fixed uses of class() (#5426)
• Force compressed buffer to be 4 bytes aligned, to keep cuda-memcheck happy (#5441)
• Remove warning for calling host function (std::max) on a GPU device (#5453)
• Fix uninitialized value bug in xgboost callback (#5463)
• Fix model dump in CLI (#5485)
• Fix out-of-bound array access in WQSummary::SetPrune() (#5493)
• Ensure that configured dmlc/build_config.h is picked up by Rabit and XGBoost, to fix build on Alpine (#5514)
• Fix a misspelled method, made i...

Release Candidate 2 of version 1.1.0

#5593

R package: xgboost_1.1.0.1.tar.gz

Release Candidate of version 1.1.0

#5593

R package: xgboost_1.1.0.1.tar.gz

1.0.2 Patch Release

This patch release applies the following patches to 1.0.0 release:

• Fix a small typo in sklearn.py that broke multiple eval metrics (#5341)
• Restore loading model from buffer. (#5360)
• Use type name for data type check. (#5364)

1.0.1 Patch Release

This release is identical to the 1.0.0 release, except that it fixes a small bug that rendered 1.0.0 incompatible with Python 3.5. See #5328.

Release 1.0.0 stable

v1.0.0 (2020.02.19)

This release marks a major milestone for the XGBoost project.

Apache-style governance, contribution policy, and semantic versioning (#4646, #4659)

• Starting with 1.0.0 release, the XGBoost Project is adopting Apache-style governance. The full community guideline is available in the doc website. Note that we now have Project Management Committee (PMC) who would steward the project on the long-term basis. The PMC is also entrusted to run and fund the project's continuous integration (CI) infrastructure (https://xgboost-ci.net).
• We also adopt the semantic versioning. See our release versioning policy.

Better performance scaling for multi-core CPUs (#4502, #4529, #4716, #4851, #5008, #5107, #5138, #5156)

• Poor performance scaling of the hist algorithm for multi-core CPUs has been under investigation (#3810). Previous effort #4529 was replaced with a series of pull requests (#5107, #5138, #5156) aimed at achieving the same performance benefits while keeping the C++ codebase legible. The latest performance benchmark results show up to 5x speedup on Intel CPUs with many cores. Note: #5244, which concludes the effort, will become part of the upcoming release 1.1.0.

Improved installation experience on Mac OSX (#4672, #5074, #5080, #5146, #5240)

• It used to be quite complicated to install XGBoost on Mac OSX. XGBoost uses OpenMP to distribute work among multiple CPU cores, and Mac's default C++ compiler (Apple Clang) does not come with OpenMP. Existing work-around (using another C++ compiler) was complex and prone to fail with cryptic diagnosis (#4933, #4949, #4969).
• Now it only takes two commands to install XGBoost: brew install libomp followed by pip install xgboost. The installed XGBoost will use all CPU cores.
• Even better, XGBoost is now available from Homebrew: brew install xgboost. See Homebrew/homebrew-core#50467.
• Previously, if you installed the XGBoost R package using the command install.packages('xgboost'), it could only use a single CPU core and you would experience slow training performance. With 1.0.0 release, the R package will use all CPU cores out of box.

Distributed XGBoost now available on Kubernetes (#4621, #4939)

• Check out the tutorial for setting up distributed XGBoost on a Kubernetes cluster.

Ruby binding for XGBoost (#4856)

New Native Dask interface for multi-GPU and multi-node scaling (#4473, #4507, #4617, #4819, #4907, #4914, #4941, #4942, #4951, #4973, #5048, #5077, #5144, #5270)

• XGBoost now integrates seamlessly with Dask, a lightweight distributed framework for data processing. Together with the first-class support for cuDF data frames (see below), it is now easier than ever to create end-to-end data pipeline running on one or more NVIDIA GPUs.
• Multi-GPU training with Dask is now up to 20% faster than the previous release (#4914, #4951).

First-class support for cuDF data frames and cuPy arrays (#4737, #4745, #4794, #4850, #4891, #4902, #4918, #4927, #4928, #5053, #5189, #5194, #5206, #5219, #5225)

• cuDF is a data frame library for loading and processing tabular data on NVIDIA GPUs. It provides a Pandas-like API.
• cuPy implements a NumPy-compatible multi-dimensional array on NVIDIA GPUs.
• Now users can keep the data on the GPU memory throughout the end-to-end data pipeline, obviating the need for copying data between the main memory and GPU memory.
• XGBoost can accept any data structure that exposes __array_interface__ signature, opening way to support other columar formats that are compatible with Apache Arrow.

Feature interaction constraint is now available with approx and gpu_hist algorithms (#4534, #4587, #4596, #5034).

Learning to rank is now GPU accelerated (#4873, #5004, #5129)

• Supported ranking objectives: NDGC, Map, Pairwise.
• Up to 2x improved training performance on GPUs.

Enable gamma parameter for GPU training (#4874, #4953)

• The gamma parameter specifies the minimum loss reduction required to add a new split in a tree. A larger value for gamma has the effect of pre-pruning the tree, by making harder to add splits.

External memory for GPU training (#4486, #4526, #4747, #4833, #4879, #5014)

• It is now possible to use NVIDIA GPUs even when the size of training data exceeds the available GPU memory. Note that the external memory support for GPU is still experimental. #5093 will further improve performance and will become part of the upcoming release 1.1.0.
• RFC for enabling external memory with GPU algorithms: #4357

Improve Scikit-Learn interface (#4558, #4842, #4929, #5049, #5151, #5130, #5227)

• Many users of XGBoost enjoy the convenience and breadth of Scikit-Learn ecosystem. In this release, we revise the Scikit-Learn API of XGBoost (XGBRegressor, XGBClassifier, and XGBRanker) to achieve feature parity with the traditional XGBoost interface (xgboost.train()).
• Insert check to validate data shapes.
• Produce an error message if eval_set is not a tuple. An error message is better than silently crashing.
• Allow using numpy.RandomState object.
• Add n_jobs as an alias of nthread.
• Roadmap: #5152

XGBoost4J-Spark: Redesigning checkpointing mechanism

• RFC is available at #4786
• Clean up checkpoint file after a successful training job (#4754): The current implementation in XGBoost4J-Spark does not clean up the checkpoint file after a successful training job. If the user runs another job with the same checkpointing directory, she will get a wrong model because the second job will re-use the checkpoint file left over from the first job. To prevent this scenario, we propose to always clean up the checkpoint file after every successful training job.
• Avoid Multiple Jobs for Checkpointing (#5082): The current method for checkpoint is to collect the booster produced at the last iteration of each checkpoint internal to Driver and persist it in HDFS. The major issue with this approach is that it needs to re-perform the data preparation for training if the user did not choose to cache the training dataset. To avoid re-performing data prep, we build external-memory checkpointing in the XGBoost4J layer as well.
• Enable deterministic repartitioning when checkpoint is enabled (#4807): Distributed algorithm for gradient boosting assumes a fixed partition of the training data between multiple iterations. In previous versions, there was no guarantee that data partition would stay the same, especially when a worker goes down and some data had to recovered from previous checkpoint. In this release, we make data partition deterministic by using the data hash value of each data row in computing the partition.

XGBoost4J-Spark: handle errors thrown by the native code (#4560)

• All core logic of XGBoost is written in C++, so XGBoost4J-Spark internally uses the C++ code via Java Native Interface (JNI). #4560 adds a proper error handling for any errors or exceptions arising from the C++ code, so that the XGBoost Spark application can be torn down in an orderly fashion.

XGBoost4J-Spark: Refine method to count the number of alive cores (#4858)

• The SparkParallelismTracker class ensures that sufficient number of executor cores are alive. To that end, it is important to query the number of alive cores reliably.

XGBoost4J: Add BigDenseMatrix to store more than Integer.MAX_VALUE elements (#4383)

Robust model serialization with JSON (#4632, #4708, #4739, #4868, #4936, #4945, #4974, #5086, #5087, #5089, #5091, #5094, #5110, #5111, #5112, #5120, #5137, #5218, #5222, #5236, #5245, #5248, #5281)

• In this release, we introduce an experimental support of using JSON for serializing (saving/loading) XGBoost models and related hyperparameters for training. We would like to eventually replace the old binary format with JSON, since it is an open format and parsers are available in many programming languages and platforms. See the documentation for model I/O using JSON. #3980 explains why JSON was chosen over other alternatives.

• To maximize interoperability and compatibility of the serialized models, we now split serialization into two parts (#4855):

  1. Model, e.g. decision trees and strictly related metadata like num_features.
  2. Internal configuration, consisting of training parameters and other configurable parameters. For example, max_delta_step, tree_method, objective, predictor, gpu_id.

  Previously, users often ran into issues where the model file produced by one machine could not load or run on another machine. For example, models trained using a machine with an NVIDIA GPU could not run on another machine without a GPU (#5291, #5234). The reason is that the old binary format saved some internal configuration that were not universally applicable to all machines, e.g. predictor='gpu_predictor'.

  Now, model saving function (Booster.save_model() in Python) will save only the model, without internal configuration. This will guarantee that your model file would be used anywhere. Internal configuration will be serialized in limited circumstances such as:

  • Multiple nodes in a distributed system exchange model details over the network.
  • Model checkpointing, to recove...