Using ScaleKernel in some models but not others (e.g. MultiTaskGP)

[Hrovatin]

I was wondering what is the reason that MultiTaskGP does not use ScaleKernel in its default covariance kernel (see here and here), while some other models use it.

[saitcakmak]

While benchmarking the new default kernel (see #2451), produced by get_covar_module_with_dim_scaled_prior, we've found it to work quite well without the scale kernel component, when the model inputs were normalized to the unit hypercube. So, by default, most models do not use a scale kernel.

As you pointed out, not all models utilize this new covar module. For example, for the fully-Bayesian model, this prior does not work that well, so that model still uses a scale-Matern kernel (@hvarfner had developed a fully Bayesian version of the new kernel as well but we haven't utilized that widely). For SingleTaskMultiFidelityGP, we need to combine two kernels together, and the use of scale kernel makes this into a weighted combination with a learnable weight.

[sdaulton]

For MultiTaskGP, the ScaleKernel isn't necessary since the scale would can be adjusted by the IndexKernel (task-task-covariance).

[Hrovatin]

Thank you both for your explanations!