EigenLoRAx: Recycling Adapters to Find Principal Subspaces for Resource-Efficient Adaptation and Inference

The Official PyTorch implementation of EigenLoRAx: Recycling Adapters to Find Principal Subspaces for Resource-Efficient Adaptation and Inference

We are still updating the code and its instructions in the coming weeks. Please watch (and leave a star if you like our work) this space for continued update.

Setup

conda env create -f environment.yml
conda activate eigenlora

Usage

In order to find the EigenLoRAs Principal Components, start with a few pretrained LoRA adapters for the same base model.

Citation

If you find EigenLoRAx useful, please consider giving a star and citation:

@misc{kaushik2025eigenloraxrecyclingadaptersprincipal,
      title={EigenLoRAx: Recycling Adapters to Find Principal Subspaces for Resource-Efficient Adaptation and Inference}, 
      author={Prakhar Kaushik and Ankit Vaidya and Shravan Chaudhari and Alan Yuille},
      year={2025},
      eprint={2502.04700},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/2502.04700}, 
}

Restriction Notice

People and teams who currently work for Adobe and its affiliates, or have done so in the recent past, are explicitly prohibited from using, modifying, or distributing this software or the idea presented in any form. If you are an Adobe affiliate but still want to use this method, please drop me an email.