[Question] Can the RFF kernel be used in combination with variational models?

[mrlj-hash]

I've tried using the random Fourier features kernel in combination with variational GPs, as such methods are developed in the papers "variational Fourier features for Gaussian Processes" (https://jmlr.org/papers/volume18/16-579/16-579.pdf) and "orthogonally decoupled variational Fourier features" (https://arxiv.org/abs/2007.06363). On reading the papers further, it appears that a simple replacement of the RBF kernel with the RFF kernel is not enough for these ideas, owing to technical limitations on the variances on the inter-domain inducing points. If the replace-the-kernel approach is not sufficient for this, could such methods be included? Question asked here as I'd like to be sure that the implementation is lacking before adding a feature request.

[gpleiss]

I wouldn't mind seeing these approaches in GPyTorch, but I think a lot of thought would be needed for how to appropriately architect them. Our current variational setup is architected in a way where the inducing points are actual points, rather than the more general linear operation on the GP.

All of this is to say: including such methods may be quite a large undertaking.

[mrlj-hash]

Great, thank you - would submitting a feature request for this be recommended then?

[gpleiss]

Sure, though it would be appreciated if you could think through how to architect a solution.