[Question] When using "SingleTaskGP", does it add OutputScale to the kernel automatically?

[mohammad-saber]

Hi, thank you very much to read my question.

I have a question about adding OutputScale prameter to the kernel when using "SingleTaskGP".

In GPyTorch, we usually add OutputScale to the Kernel using gpytorch.kernels.ScaleKernel in the model definition as follows:

class ExactGPModel(gpytorch.models.ExactGP):
    def __init__(self, train_x, train_y, likelihood):
        super(ExactGPModel, self).__init__(train_x, train_y, likelihood)
        self.mean_module = gpytorch.means.ConstantMean()
        self.covar_module = gpytorch.kernels.ScaleKernel(gpytorch.kernels.RBFKernel())

    def forward(self, x):
        mean_x = self.mean_module(x)
        covar_x = self.covar_module(x)
        return gpytorch.distributions.MultivariateNormal(mean_x, covar_x)

I was wondering if "SingleTaskGP" class multiplies kernel by OutputScale automatically, or I should define kernel like below:

import torch
from gpytorch.kernels import ScaleKernel, RBFKernel
from botorch.models import SingleTaskGP

train_x = torch.rand(10, 2)
train_y = 1 - torch.norm(train_x - 0.5, dim=-1, keepdim=True)

kernel = ScaleKernel(RBFKernel())
model = SingleTaskGP(train_x, train_y, covar_module=kernel)   

[saitcakmak]

Hi @mohammad-saber. The default kernel of the SingleTaskGP is defined here

botorch/botorch/models/gp_regression.py

Lines 123 to 133 in 70143a7

	if covar_module is None:
	self.covar_module = ScaleKernel(
	MaternKernel(
	nu=2.5,
	ard_num_dims=transformed_X.shape[-1],
	batch_shape=self._aug_batch_shape,
	lengthscale_prior=GammaPrior(3.0, 6.0),
	),
	batch_shape=self._aug_batch_shape,
	outputscale_prior=GammaPrior(2.0, 0.15),
	)

It's an ARD Matern 5/2 kernel with an output scale.

Looking more closely, your question is about passing in a custom kernel. In this case, it will be used as-is, so you should define it with a ScaleKernel as in your example.