Transformer_Pytorch

Transformer的pytorch实现 (attention-is-all-you-need)

运行环境

• python2.7
• pytorch1.0.1

程序执行

• python Transformer.py 可以看到如下过程

('step: ', 850, '| time: 10.69', '| loss: 211.348', '| target: ', '16/Feb/2010', '| inference: ', '16/Feb/2016')
('step: ', 900, '| time: 11.46', '| loss: 203.111', '| target: ', '23/Sep/2012', '| inference: ', '23/Sep/2018')

Referance

• https://github.com/MorvanZhou/NLP-Tutorials/blob/master/Transformer.py tensorflow版本
• https://juejin.im/post/5b9f1af0e51d450e425eb32d#comment 原理讲解的很好