Implementation & distributed training of GPT2 (may be gpt3)
- the official GPT-2 TensorFlow implementation released by OpenAI: https://github.com/openai/gpt-2/blob/master/src/model.py
- huggingface/transformers PyTorch implementation: https://github.com/huggingface/transformers/blob/main/src/transformers/models/gpt2/modeling_gpt2.py
- Andra Karpathy nanoGPT implementation: https://github.com/karpathy/nanoGPT