Paper-Implementing_Attention-Is-All-You-Need

Citation

@article{vaswani2017attention,
  title={Attention is all you need},
  author={Vaswani, A},
  journal={Advances in Neural Information Processing Systems},
  year={2017}
}

Architecture

Note: All components of the Transformer network have been implemented, except for positional encoding.

Next step: As it is hard for me to train tranformer network from scartch, I will fine-tune pretrained transformers like BERT.