From prototype to production — an end-to-end ecosystem for building & fine-tuning production-grade LLMs
🛠️ Some parts are stable and ready for you to explore, others are actively evolving or experimental.
- Notebooks: Interactive tutorials for hands-on learning
- Modules: Modular mini-projects and training scripts
- MyLLM Core Framework: Pure PyTorch, clean, extensible, actively developed
- MetaBot Chatbot: Coming soon — a chatbot that explains itself!
Jump in, experiment, break stuff, learn! This repo is a sandbox for us all.
- Check out
notebooks/for step-by-step transformer tutorials. - Run, tweak, and observe model internals in action.
git clone https://github.com/silvaxxx1/MyLLM101.git
cd MyLLM101
pip install -r requirements.txt
jupyter notebook notebooks/0.0.WELCOME.ipynbPro tip: Modify the attention mask code and see what changes in output!
- Organize reusable components in
modules/. - Train small GPTs from scratch quickly:
python modules/train_gpt.py --config configs/basic.yml- Pure PyTorch native transformer models with minimal dependencies.
- Inspired by LitGPT and Hugging Face for clean APIs and modularity.
- Designed for deep understanding and flexibility in research and production.
from myllm import LLM, SFTTrainer, DPOTTrainer, PPOTrainer, Quantizer
# Load a pretrained or fine-tuned model
llm = LLM.load("my_model_checkpoint")
# Generate text with flexible options
output = llm.generate("Once upon a time, in a world of AI,")
print(output)
# Supervised Fine-Tuning (SFT)
sft = SFTTrainer(model=llm, dataset=my_training_data)
sft.train(epochs=3, batch_size=16)
# Direct Preference Optimization (DPO)
dpo = DPOTTrainer(model=llm, dataset=my_preference_data)
dpo.train(epochs=5)
# Proximal Policy Optimization (PPO) for RLHF
ppo = PPOTrainer(model=llm, environment=my_custom_env)
ppo.train(iterations=10)
# Quantization for efficient inference
quantizer = Quantizer(model=llm)
llm_quantized = quantizer.apply(precision="int8")Experiment combining features: What happens if you quantize before fine-tuning?
- A chatbot that not only talks but explains how it was built.
- Built with MyLLM + Gradio UI for smooth interaction.
- Help build this next milestone!
- Run a notebook, tweak learning rates, observe training dynamics.
- Train a mini GPT, generate text, try making it rhyme!
- Write your own prompts and experiment with completions.
- Fork the repo, add a new trainer (PPO is waiting for you!).
- Help build the Gradio UI for MetaBot — ideas welcome!
| Status | Milestone | Details |
|---|---|---|
| ✅ | Interactive Notebooks | Learn LLM fundamentals hands-on |
| ✅ | Modular Mini-Projects | Build reusable, composable components |
| ⚙️ | MyLLM Core Framework | Fine-tuning, DPO, PPO, quantization |
| 🛠 | MetaBot Chatbot + Gradio UI | Interactive chatbot & deployment |
- To learn deeply by building transformers from scratch
- To share an open, transparent development journey
- To demystify transformer internals and fine-tuning techniques
- To create a scalable, extensible platform for experimentation
- Umar Jamil — Practical LLM tutorials
- Andrej Karpathy — NanoGPT minimalism
- Sebastian Raschka — Deep transformer insights
Their work motivated this project.
Jump in! Experiment! Ask questions! Build your own Meta_Bot!