Merge branch 'pytorch:main' into main

Author: seashoo

.github/workflows/nightly.yml

@@ -75,9 +75,8 @@ jobs:
       contents: write
 
   run_tutorials:
-    name: Run tutorials without smoke test on latest PyTorch / GPyTorch / Ax
+    name: Run tutorials without smoke test on latest PyTorch / GPyTorch
     uses: ./.github/workflows/reusable_tutorials.yml
     with:
       smoke_test: false
       use_stable_pytorch_gpytorch: false
-      use_stable_ax: false

.github/workflows/publish_website.yml

@@ -47,11 +47,8 @@ jobs:
     - name: Install dependencies
       env:
         ALLOW_LATEST_GPYTORCH_LINOP: true
-        ALLOW_BOTORCH_LATEST: true  # Allow Ax to install w/ new BoTorch release.
       run: |
         uv pip install ."[dev, tutorials]"
-        # There may not be a compatible Ax uv pip version, so we use the development version.
-        uv pip install git+https://github.com/facebook/Ax.git
     - if: ${{ inputs.new_version }}
       name: Create new docusaurus version
       run: |

.github/workflows/reusable_tutorials.yml

@@ -9,9 +9,6 @@ on:
       use_stable_pytorch_gpytorch:
         required: true
         type: boolean
-      use_stable_ax:
-        required: true
-        type: boolean
   workflow_call:
     inputs:
       smoke_test:
@@ -22,10 +19,6 @@ on:
         required: false
         type: boolean
         default: false
-      use_stable_ax:
-        required: false
-        type: boolean
-        default: false
 
 jobs:
   tutorials:
@@ -66,19 +59,6 @@ jobs:
         ALLOW_LATEST_GPYTORCH_LINOP: true
       run: |
         uv pip install .[tutorials]
-    - if: ${{ !inputs.use_stable_ax }}
-      name: Install latest Ax
-      env:
-        # This is so Ax's setup doesn't install a pinned BoTorch version.
-        ALLOW_BOTORCH_LATEST: true
-      run: |
-        uv pip install git+https://github.com/facebook/Ax.git
-    - if: ${{ inputs.use_stable_ax }}
-      name: Install stable Ax
-      env:
-        ALLOW_BOTORCH_LATEST: true
-      run: |
-        uv pip install ax-platform --no-binary ax-platform
     - if: ${{ inputs.smoke_test }}
       name: Run tutorials with smoke test
       run: |

.github/workflows/test_stable.yml

@@ -52,18 +52,16 @@ jobs:
       run: |
         pytest -ra test_community/ --cov botorch_community/ --cov-report term-missing --cov-report xml:botorch_community_cov.xml
 
-  run_tutorials_stable_w_latest_ax:
-    name: Run tutorials without smoke test on min req. versions of PyTorch & GPyTorch and latest Ax
+  run_tutorials_stable:
+    name: Run tutorials without smoke test on min req. versions of PyTorch & GPyTorch
     uses: ./.github/workflows/reusable_tutorials.yml
     with:
       smoke_test: false
       use_stable_pytorch_gpytorch: true
-      use_stable_ax: false
 
   run_tutorials_stable_smoke_test:
-    name: Run tutorials with smoke test on min req. versions of PyTorch & GPyTorch and latest Ax
+    name: Run tutorials with smoke test on min req. versions of PyTorch & GPyTorch
     uses: ./.github/workflows/reusable_tutorials.yml
     with:
       smoke_test: true
       use_stable_pytorch_gpytorch: true
-      use_stable_ax: false

.github/workflows/tutorials_smoke_test.yml

@@ -10,9 +10,8 @@ on:
 
 jobs:
   run_tutorials_with_smoke_test:
-    name: Run tutorials with smoke test on latest PyTorch / GPyTorch / Ax
+    name: Run tutorials with smoke test on latest PyTorch / GPyTorch
     uses: ./.github/workflows/reusable_tutorials.yml
     with:
       smoke_test: true
       use_stable_pytorch_gpytorch: false
-      use_stable_ax: false

CHANGELOG.md

@@ -2,6 +2,57 @@
 
 The release log for BoTorch.
 
+## [0.14.0] -- May 6, 2025
+
+#### Highlights
+* Prior Fitted Network (PFN) surrogate model integration (#2784).
+* Variational Bayesian last-layer models as surrogate `Model`s (#2754).
+* Probabilities of feasibility for classifier-based constraints in acquisition functions (#2776).
+
+#### New Features
+* Helper for evaluating feasibility of candidate points (#2733).
+  * Check for feasibility in `gen_candidates_scipy` and error out for infeasible candidates (#2737).
+  * Return a feasible candidate if there is one and `return_best_only=True` (#2778).
+* Allow for observation noise without provided `evaluation_mask` mask in `ModelListGP` (#2735).
+* Implement incremental `qLogNEI` via `incremental` argument to `qLogNoisyExpectedImprovement` (#2760).
+* Add utility for computing AIC/BIC/MLL from a model (#2785).
+* New test functions:
+  * Multi-fidelity test functions with discrete fidelities (#2796).
+  * Keane bump function (#2802).
+  * Mixed Ackley test function (#2830).
+  * LABS test function (#2832).
+* Add parameter types to test functions to support problems defined in mixed / discrete spaces (#2809).
+  * Add input validation to test functions (#2829).
+* Add `[q]LogProbabilityOfFeasibility` acquisition functions (#2815).
+
+#### Bug Fixes
+* Remove hard-coded `dtype` from `best_f` buffers (#2725).
+* Fix `dtype/nan` issue in `StratifiedStandardize` (#2757).
+* Properly handle observed noise in `AdditiveMapSaasSingleTaskGP` with outcome transforms (#2763).
+* Do not count STOPPED (due to specified budget) as a model fitting failure (#2767).
+* Ensure that `initialize_q_batch` always includes the maximum value when called in batch mode (#2773).
+* Fix posterior with observation noise in batched MTGP models (#2782).
+* Detach tensor in `gen_candidates_scipy` to avoid test failure due to new warning (#2797).
+* Fix batch computation in Pivoted Cholesky (#2823).
+
+#### Other Changes
+* Add optimal values for synthetic contrained optimization problems (#2730).
+  * Update `max_hv` and reference point for Penicillin problem (#2771).
+  * Add optimal value to SpeedReducer problem (#2799).
+* Update `nonlinear_constraint_is_feasible` to return a boolean tensor (#2731).
+* Restructure sampling methods for info-theoretic acquisition functions (#2753).
+* Prune baseline points in `qLogNEI` by default (#2762).
+* Misc updates to MES-based acqusition functions (#2769).
+* Pass option to reset submodules in train method for fully Bayesian models (#2783).
+* Put outcome transforms into train mode in model constructors (#2817).
+* `LogEI`: select `cache_root` based on model support (#2820).
+* Remove Ax dependency from BoTorch tutorials and reference Ax tutorials instead (#2839).
+
+#### Deprecations and removals
+* Remove deprecated `gp_sampling` module (#2768).
+* Remove `qMultiObjectiveMaxValueEntropy` acquisition function (#2800).
+* Remove model converters (#2801).
+
 
 ## [0.13.0] -- Feb 3, 2025
 

botorch_community/models/utils/prior_fitted_network.py

@@ -28,16 +28,12 @@ class ModelPaths(Enum):
 
     pfns4bo_hebo = (
         "https://github.com/automl/PFNs4BO/raw/refs/heads/main/pfns4bo"
-        "/final_models/hebo_morebudget_9_unused_features_3_userpriorperdim2_8.pt.gz"
+        "/final_models/model_hebo_morebudget_9_unused_features_3.pt.gz"
     )
     pfns4bo_bnn = (
         "https://github.com/automl/PFNs4BO/raw/refs/heads/main/pfns4bo"
         "/final_models/model_sampled_warp_simple_mlp_for_hpob_46.pt.gz"
     )
-    pfns4bo_hebo_userprior = (
-        "https://github.com/automl/PFNs4BO/raw/refs/heads/main/pfns4bo"
-        "/final_models/hebo_morebudget_9_unused_features_3_userpriorperdim2_8.pt.gz"
-    )
 
 
 def download_model(

docs/botorch_and_ax.md

@@ -42,8 +42,8 @@ of surrogate model, or a new type of acquisition function, but leave the rest of
 the the Bayesian Optimization loop untouched. It is then straightforward to plug
 your custom BoTorch model or acquisition function into Ax to take advantage of
 Ax's various loop control APIs, as well as its powerful automated metadata
-management, data storage, etc. See the
-[Using a custom BoTorch model in Ax](tutorials/custom_botorch_model_in_ax)
+management, data storage, etc. See Ax's
+[Modular BoTorch tutorial](https://ax.dev/docs/tutorials/modular_botorch/)
 tutorial for more on how to do this.
 
 

docs/models.md

@@ -168,9 +168,9 @@ configurable model class whose implementation is difficult to understand.
 
 Instead, we advocate that users implement their own models to cover more
 specialized use cases. The light-weight nature of BoTorch's Model API makes this
-easy to do. See the
-[Using a custom BoTorch model in Ax](tutorials/custom_botorch_model_in_ax)
-tutorial for an example.
+easy to do. See Ax's
+[Modular BoTorch tutorial](https://ax.dev/docs/tutorials/modular_botorch/)
+tutorial for an example for this and how to use such a custom model in Ax.
 
 The BoTorch `Model` interface is light-weight and easy to extend. The only
 requirement for using BoTorch's Monte-Carlo based acquisition functions is that

docs/tutorials/index.mdx

@@ -4,27 +4,32 @@ title: BoTorch Tutorials
 The tutorials here will help you understand and use BoTorch in
 your own work. They assume that you are familiar with both
 Bayesian optimization (BO) and PyTorch.
-
-If you are new to BO, we recommend you start with the
+* If you are new to BO, we recommend you start with the
 [Ax docs](https://ax.dev/docs/bayesopt) and the
 following
 [tutorial paper](https://arxiv.org/abs/1807.02811).
-
-If you are new to PyTorch, the easiest way to get started is
+* If you are new to PyTorch, the easiest way to get started is
 with the [What is PyTorch?](https://pytorch.org/tutorials/beginner/blitz/tensor_tutorial.html#sphx-glr-beginner-blitz-tensor-tutorial-py)
 tutorial.
 
-The BoTorch tutorials are grouped into the following four areas.
-
 
 <h4>Using BoTorch with Ax</h4>
-These tutorials give you an overview of how to leverage
-[Ax](https://ax.dev), a platform for sequential
-experimentation, in order to simplify the management of your BO
-loop. Doing so can help you focus on the main aspects of BO
-(models, acquisition functions, optimization of acquisition
-functions), rather than tedious loop control. See our
-[Documentation](/docs/botorch_and_ax)
+_For practitioners_ who are interested in running experiments
+to optimize various objectives using Bayesian optimization,
+we recommend using [Ax](https://ax.dev) rather than BoTorch.
+[Ax](https://ax.dev) provides a user-friendly interface for
+experiment configuration and orchestration, while choosing an
+appropriate Bayesian optimization algorithm to optimize the
+given objective, following BoTorch best practices.
+
+_For researchers_ who are interested in running experiments with
+their custom BoTorch models and acquisition functions,
+[Ax](https://ax.dev)'s Modular BoTorch Interface offers a convenient
+way to leverage custom BoTorch objects while utilizing
+[Ax](https://ax.dev) experiment configuration and orchestration. Check out
+[Modular BoTorch tutorial](https://ax.dev/docs/tutorials/modular_botorch/)
+to learn how to use custom BoTorch objects in Ax!
+See [this documentation](/docs/botorch_and_ax)
 for additional information.
 
 <h4>Full Optimization Loops</h4>

setup.py

@@ -17,10 +17,8 @@
 TEST_REQUIRES = ["pytest", "pytest-cov", "requests"]
 FMT_REQUIRES = ["flake8", "ufmt", "flake8-docstrings"]
 TUTORIALS_REQUIRES = [
-    "ax-platform",
     "cma",
     "jupyter",
-    "kaleido",
     "matplotlib",
     "memory_profiler",
     "papermill",

tutorials/bo_with_warped_gp/bo_with_warped_gp.ipynb

@@ -8,7 +8,7 @@
         "\n",
         "In this tutorial, we illustrate how to use learned input warping functions for robust Bayesian Optimization when the outcome may be non-stationary functions. When the lengthscales are non-stationarity in the raw input space, learning a warping function that maps raw inputs to a warped space where the lengthscales are stationary can be useful, because then standard stationary kernels can be used to effectively model the function.\n",
         "\n",
-        "In general, for a relatively simple setup (like this one), we recommend using [Ax](https://ax.dev), since this will simplify your setup (including the amount of code you need to write) considerably. See the [Using BoTorch with Ax](/docs/tutorials/custom_botorch_model_in_ax) tutorial. To use input warping with `MODULAR_BOTORCH`, we can pass the `warp_tf`, constructed as below, by adding `input_transform=warp_tf` argument to the `Surrogate(...)` call. \n",
+        "In general, for a relatively simple setup (like this one), we recommend using [Ax](https://ax.dev), since this will simplify your setup (including the amount of code you need to write) considerably. See Ax's [Modular BoTorch tutorial](https://ax.dev/docs/tutorials/modular_botorch/) tutorial. To use input warping with `MODULAR_BOTORCH`, we can pass the `warp_tf`, constructed as below, by adding `input_transform=warp_tf` argument to the `Surrogate(...)` call. \n",
         "\n",
         "We consider use a Kumaraswamy CDF as the class of input warping function and learn the concentration parameters ($a>0$ and $b>0$). Kumaraswamy CDFs are quite flexible and map inputs in [0, 1] to outputs in [0, 1]. This work follows the Beta CDF input warping proposed by Snoek et al., but replaces the Beta distribution Kumaraswamy distribution, which has a *differentiable* and closed-form CDF. \n",
         "   \n",

tutorials/closed_loop_botorch_only/closed_loop_botorch_only.ipynb

@@ -11,7 +11,7 @@
         "\n",
         "In this tutorial, we illustrate how to implement a simple Bayesian Optimization (BO) closed loop in BoTorch.\n",
         "\n",
-        "In general, we recommend for a relatively simple setup (like this one) to use Ax, since this will simplify your setup (including the amount of code you need to write) considerably. See the [Using BoTorch with Ax](/docs/tutorials/custom_botorch_model_in_ax) tutorial.\n",
+        "In general, we recommend for a relatively simple setup (like this one) to use Ax, since this will simplify your setup (including the amount of code you need to write) considerably. See Ax's [Modular BoTorch tutorial](https://ax.dev/docs/tutorials/modular_botorch/) tutorial.\n",
         "\n",
         "However, you may want to do things that are not easily supported in Ax at this time (like running high-dimensional BO using a VAE+GP model that you jointly train on high-dimensional input data). If you find yourself in such a situation, you will need to write your own optimization loop, as we do in this tutorial.\n",
         "\n",

tutorials/constrained_multi_objective_bo/constrained_multi_objective_bo.ipynb

@@ -11,7 +11,7 @@
         "\n",
         "In this tutorial, we illustrate how to implement a constrained multi-objective (MO) Bayesian Optimization (BO) closed loop in BoTorch.\n",
         "\n",
-        "In general, we recommend using [Ax](https://ax.dev) for a simple BO setup like this one, since this will simplify your setup (including the amount of code you need to write) considerably. See [here](https://ax.dev/docs/tutorials/multiobjective_optimization.html) for an Ax tutorial on MOBO. If desired, you can use a custom BoTorch model in Ax, following the [Using BoTorch with Ax](/docs/tutorials/custom_botorch_model_in_ax) tutorial. Given a `MultiObjective`, Ax will default to the $q$NEHVI acquisiton function. If desired, this can also be customized by adding `\"botorch_acqf_class\": <desired_botorch_acquisition_function_class>,` to the `model_kwargs`.\n",
+        "In general, we recommend using [Ax](https://ax.dev) for a simple BO setup like this one, since this will simplify your setup (including the amount of code you need to write) considerably. See [here](https://ax.dev/docs/tutorials/multiobjective_optimization.html) for an Ax tutorial on MOBO. If desired, you can use a custom BoTorch model in Ax, following Ax's [Modular BoTorch tutorial](https://ax.dev/docs/tutorials/modular_botorch/) tutorial. Given a `MultiObjective`, Ax will default to the $q$NEHVI acquisiton function. If desired, this can also be customized by adding `\"botorch_acqf_class\": <desired_botorch_acquisition_function_class>,` to the `model_kwargs`.\n",
         "\n",
         "We use the parallel ParEGO ($q$ParEGO) [1] and parallel Noisy Expected Hypervolume Improvement ($q$NEHVI) [2]  acquisition functions to optimize a synthetic C2-DTLZ2 test function with $M=2$ objectives, $V=1$ constraint, and $d=4$ parameters. The two objectives are\n",
         "$$f_1(\\mathbf x) = (1+ g(\\mathbf x_M))\\cos\\big(\\frac{\\pi}{2}x_1\\big)$$\n",