Transformer-Language-Modeling demonstration of how to build a transformer language model from scratch virtual env python -m venv transformer_venv How to run Part 0 python letter_counting.py --task BEFOREAFTER Part 1 python letter_counting.py Part 2 python lm.py python lm.py --model NEURAL Relevant links https://pytorch.org/docs/stable/generated/torch.nn.TransformerEncoder.html http://mattmahoney.net/dc