awd-lstm-att-lm

awd-lstm-att-lm is based on awd-lstm-lm, all unmodified code is under its original license.

Running

Setup

conda create --name my_env python=3.7
conda activate my_env
conda install pytorch=0.4.1 cuda90 -c pytorch

Models

Download trained models:

PTB_att.pt

PTB_att_pruned.pt

WT2_att.pt

WT2_att_pruned.pt

Data

To get data run:

./getdata.sh

Inference

For PTB run:

python infer_new.py --data data/penn --load PTB_att.pt

For PTB pruned run:

python infer_new.py --data data/penn --load PTB_att_pruned.pt

For WikiText-2 run:

python infer_new.py --data data/wikitext-2 --load WT2_att.pt

For WikiText-2 pruned run:

python infer_new.py --data data/wikitext-2 --load WT2_att_pruned.pt

Results

		Number of params	test ppl	valid ppl
PTB	awd-lstm-att-lm	28.7M	45.22	46.50
	awd-lstm-att-lm + pruning	18.5M	45.94	47.29
WT-2	awd-lstm-att-lm	47.1M	35.27	36.97
	awd-lstm-att-lm + pruning	31.9M	38.31	40.07