@@ -41,3 +41,34 @@ To obtain metric scores on the best model run `main.py` with *--train=False* or
41
41
``` console
42
42
❱❱❱ python main.py --task_id 1
43
43
```
44
+
45
+ # Analysis
46
+
47
+ ## Trainable Paramters
48
+ The following numbers are reported using * embedding_size 256* , * batch_size 64* , * hops 6*
49
+
50
+ | | Task 1 | Task 2 | Task 3 | Task 4 | Task 5 | CamRest | SMD |
51
+ | ------ | ------ | ------ | ------ | ------ | ------ | ------ | ------ |
52
+ | ** BossNet** | 1204993 | 1198849 | 1478913 | 1715457 | 1943809 | 1968129 | 2187265 |
53
+ | ** Mem2Seq** | 776030 | 780127 | 3426789 | 4508397 | 5049201 | 5274536 | 6880560 |
54
+ | ** Seq2Seq + Copy** | 6379859 | 6378321 | 6870481 | 7083494 | 7167315 | 7223452 | 7504137 |
55
+ | ** Seq2Seq** | 6905172 | 6903634 | 7395794 | 7608807 | 7692628 | 7748765 | 8029450 |
56
+
57
+ ## Running Times
58
+ The following numbers are reported using * embedding_size 128* , * batch_size 64* , * hops 3*
59
+ Times are reported as ` sec. per train epoch (avg. no. of epochs till convergence) **total train time** `
60
+
61
+ | | Task 1 | Task 2 | Task 3 | Task 4 | Task 5 | CamRest | SMD |
62
+ | ------ | ------ | ------ | ------ | ------ | ------ | ------ | ------ |
63
+ | ** BossNet** | 38.1 (15) ** 571.5** | 65.2 (10) ** 652.0** | 142.4 (25) ** 3560** | 16.9 (2) ** 33.8** | 231.3 (6) ** 1387.8** | 113.5 (6) ** 681** | 1252 (10) ** 12520** |
64
+ | ** Mem2Seq** | 10 (100) ** 1000** | 32 (30) ** 960** | 51 (90) ** 4590** | 4 (10) ** 40** | 136 (60) ** 8160** | 22 (40) ** 880** | 81 (40) ** 3240** |
65
+
66
+ ## Hyperparameters
67
+
68
+ | | Task 1 | Task 2 | Task 3 | Task 4 | Task 5 | CamRest | SMD |
69
+ | ------ | ------ | ------ | ------ | ------ | ------ | ------ | ------ |
70
+ | Learning Rate | 0.001 | 0.001 | 0.005 | 0.001 | 0.0005 | 0.0005 | 0.0005 |
71
+ | Hops | 1 | 1 | 3 | 1 | 3 | 6 | 3 |
72
+ | Embedding Size | 128 | 128 | 128 | 128 | 256 | 256 | 256 |
73
+ | Disentangle Loss Weight | 1.0 | 1.0 | 1.5 | 1.0 | 1.0 | 1.0 | 1.0 |
74
+ | DLD | 0.2 | 0.2 | 0.2 | 0.2 | 0.2 | 0.2 | 0.1 |
0 commit comments