Train Small, Infer Large
Memory-Efficient LoRA Training for LLMs

🚀 15.81×～16.95× Parameter Reduction ⬇️

LoRAM is a memory-efficient LoRA training method for cost-effective performance gains by
training low-rank matrices on a pruned model and merging them for inference on the original model.

Jun Zhang¹, Jue Wang¹, Huan Li¹, Lidan Shou¹, Ke Chen¹,
Yang You², Guiming Xie³, Xuejian Gong³, Kunlong Zhou³

¹ Zhejiang University, ² National University of Singapore, ³ OPPO AI Center

---

📌 Overview

---

🔥 Features

✅ Train LoRA on a pruned model to reduce memory footprint
✅ Recover LoRA for high-quality full model inference

---

🛠 Installation

Clone the repository and install dependencies:

git clone https://github.com/your-repo/LoRAM.git
cd LoRAM/loram

---

🙌 Acknowledgments

🤝 Institutional Collaboration

This project was made possible thanks to a collaboration with:
       

🤝 Tool Contributions

Shout out to LLM-Pruner and SparseGPT!
LoRAM leverages these tools, and we appreciate their contributions to the research community.

---

📖 Citation

If you find the resources in this repository useful, please cite our paper:

@inproceedings{
zhang2025train,
title={Train Small, Infer Large: Memory-Efficient Lo{RA} Training for Large Language Models},
author={Jun Zhang and Jue WANG and Huan Li and Lidan Shou and Ke Chen and Yang You and Guiming Xie and Xuejian Gong and Kunlong Zhou},
booktitle={The Thirteenth International Conference on Learning Representations},
year={2025},
url={https://openreview.net/forum?id=s7DkcgpRxL}
}

---