Skip to content

shunzh/llm.ipynb

Folders and files

NameName
Last commit message
Last commit date

Latest commit

ย 

History

10 Commits
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 

Repository files navigation

๐Ÿง  Deep Learning from Scratch

Minimal PyTorch implementations of core deep learning components, built from scratch to understand how they work.

๐Ÿ““ Notebooks

  • llm.ipynb
    A minimal decoder-only Transformer for language modeling, with code for architecture, training, and generation.

  • adamw.ipynb
    A from-scratch implementation of the AdamW optimizer, comparing it against L2-regularized Adam on a toy regression task.

๐Ÿ’ก Tip: Run these notebooks directly in Google Colab โ€” no setup required.

About

Implementing a Decoder-Only Transformer from Scratch.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published