The Onset of Variance in Kernel and Neural Network Regression

This is the code to reproduce the experiments and figures in the ICLR 2022 paper "The Onset of Variance Limited Behavior for Networks in the Lazy and Rich Regimes" https://arxiv.org/abs/2212.12147

The principal script is Scripts/pn_sweeps.py. This allows one to sweep over width, depth, dataset size, polynomial degree, initialization seed, dataset seed, number of ensembles.

This also requires the kernel_generalization package to be installed with

!pip install -q git+https://github.com/Pehlevan-Group/kernel-generalization

The notebooks to reproduce the major figures are in PaperFigures