Fairseq with transformer evolution

This is the codebase we use in the following paper, the code is based on fairseq:

• The Devil in Linear Transformer
• Toeplitz Neural Network for Sequence Modeling
• Hierarchically Gated Recurrent Neural Network for Sequence Modeling, NeurIPS 2023 spotlight

Install

pip install --editable ./
git clone https://github.com/Doraemonzzz/hgru-pytorch.git
pip install -e .