Skip to content

Commit 153d26a

Browse files
Added Analysis Section to README
1 parent c46e729 commit 153d26a

File tree

1 file changed

+31
-0
lines changed

1 file changed

+31
-0
lines changed

README.md

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -41,3 +41,34 @@ To obtain metric scores on the best model run `main.py` with *--train=False* or
4141
```console
4242
❱❱❱ python main.py --task_id 1
4343
```
44+
45+
# Analysis
46+
47+
## Trainable Paramters
48+
The following numbers are reported using *embedding_size 256*, *batch_size 64*, *hops 6*
49+
50+
| | Task 1 | Task 2 | Task 3 | Task 4 | Task 5 | CamRest | SMD |
51+
| ------ | ------ | ------ | ------ | ------ | ------ | ------ | ------ |
52+
| **BossNet** | 1204993 | 1198849 | 1478913 | 1715457 | 1943809 | 1968129 | 2187265 |
53+
| **Mem2Seq** | 776030 | 780127 | 3426789 | 4508397 | 5049201 | 5274536 | 6880560 |
54+
| **Seq2Seq + Copy** | 6379859 | 6378321 | 6870481 | 7083494 | 7167315 | 7223452 | 7504137 |
55+
| **Seq2Seq** | 6905172 | 6903634 | 7395794 | 7608807 | 7692628 | 7748765 | 8029450 |
56+
57+
## Running Times
58+
The following numbers are reported using *embedding_size 128*, *batch_size 64*, *hops 3*
59+
Times are reported as ` sec. per train epoch (avg. no. of epochs till convergence) **total train time** `
60+
61+
| | Task 1 | Task 2 | Task 3 | Task 4 | Task 5 | CamRest | SMD |
62+
| ------ | ------ | ------ | ------ | ------ | ------ | ------ | ------ |
63+
| **BossNet** | 38.1 (15) **571.5** | 65.2 (10) **652.0** | 142.4 (25) **3560** | 16.9 (2) **33.8** | 231.3 (6) **1387.8** | 113.5 (6) **681** | 1252 (10) **12520** |
64+
| **Mem2Seq** | 10 (100) **1000** | 32 (30) **960** | 51 (90) **4590** | 4 (10) **40** | 136 (60) **8160** | 22 (40) **880** | 81 (40) **3240** |
65+
66+
## Hyperparameters
67+
68+
| | Task 1 | Task 2 | Task 3 | Task 4 | Task 5 | CamRest | SMD |
69+
| ------ | ------ | ------ | ------ | ------ | ------ | ------ | ------ |
70+
| Learning Rate | 0.001 | 0.001 | 0.005 | 0.001 | 0.0005 | 0.0005 | 0.0005 |
71+
| Hops | 1 | 1 | 3 | 1 | 3 | 6 | 3 |
72+
| Embedding Size | 128 | 128 | 128 | 128 | 256 | 256 | 256 |
73+
| Disentangle Loss Weight | 1.0 | 1.0 | 1.5 | 1.0 | 1.0 | 1.0 | 1.0 |
74+
| DLD | 0.2 | 0.2 | 0.2 | 0.2 | 0.2 | 0.2 | 0.1 |

0 commit comments

Comments
 (0)