Implementing LoRA from Scratch

This repo finetunes Llama3.2 models on Alpaca dataset with a custom, simplified implementation of LoRA. To make my life easier, I have used the model definition of Llama3.2 and, dataset preparation from @rasbt

This project would have taken lot longer without @rasbt and his intuitive implementations, so many thanks to @rasbt.

How to use:

Getting started:

Reading LoRA's paper

LoRA Toy example:

Use the lora_toy_example.ipynb notebook to understand need of LoRA and it's basic building blocks.

Diving deep:

To use my customized implementation of LoRA, please follow below-mentioned steps:

• Download alpaca dataset from here
• Prepare the dataset using llama3_2_lora/data/prepare_alpaca_dataset.py script.
• Download the Llama3.2 checkpoint from Huggingface
• (Optional) Understand the model architecture from llama3_2_lora/model/llama3_2_standalone.py
• Understand the custom LoRA implementation from llama3_2_lora/lora/lora.py
• Finetune the model using llama3_2_lora/finetune_lora.py

Expected output, runtime and memory requirement:

At the end of finetuning, you will find several checkpoints in llama3_2_lora/checkpoint folder. The finetune script can be configured to save intermediate checkpoints at any interval. This script take roughly ~3 to 4 hours to complete on RTX 3090 and uses ~11-13G of VRAM.

ToDo:

Add finetuning with QLoRA.