Experimental implementation of "Looped Transformers are Better at Learning Learning Algorithms" showing superior performance with 12x fewer parameters. Includes complete environment setup, pre-trained weights, and extensive experiments comparing Looped TFs vs traditional Transformers for In-Context Learning.
machine-learning deep-learning transformers pytorch research-paper model-compression pytorch-implementation looped-transformers
-
Updated
Aug 5, 2024 - Jupyter Notebook