Implement a Transformer from Scratch using JAX

This project demonstrates the implementation of a Transformer model from scratch using JAX. The Transformer model, introduced in the paper "Attention Is All You Need" by Vaswani et al., has revolutionized the field of natural language processing.

Repository Structure

• data/: Contains raw and processed data.
• models/: Contains the implementation of the Transformer model and attention mechanisms.
• notebooks/: Jupyter notebooks for data preparation, training, evaluation, and visualization.
• scripts/: Python scripts for training, evaluation, and visualization.
• utils/: Utility functions for data loading and visualization.
• requirements.txt: List of dependencies.
• README.md: Project overview and instructions.
• LICENSE: Project license.

Acknowledgments

"Attention Is All You Need" by Vaswani et al. The PyTorch community for providing excellent resources and support.