Alex remove lemma rebased #1518

AngledLuffa · 2025-10-07T23:31:21Z

Summarization - WIP draft

…troduce p_gen and coverage

…dex map from inputs to extended vocab map IDs. Also computes largest number of OOV words.

…firming correct implementation

…tatements

…edding as the UNK word emb

…egin method for training model

…xceptions if they're missing.

…M, and after attention layer and the final linear layer in the decoder

…l.py

…clipping. print gradient norms

…cture

…lization for linear layers

…one, not the object itself

…ry end of the distribution calculation

…on sections

…layer LSTMs

… decoded examples

… beam search

…aries.

…se padding too

SecroLoL and others added 30 commits October 7, 2025 16:30

Begin baseline model; skeleton code

28b605a

Initial baseline + pointergen model. TODO fix dimensionalities and in…

368280e

…troduce p_gen and coverage

Add baseline model (seq2seq no coverage, no pgen)

5bdbc96

Add constants for model building and args loading

e8a0407

Delete out logging statements

6390a53

Add printing settings to see deeper into tensors

d6811f2

Add masking to the attention lyaer

94b2056

add embedding dim to the decoder linear layer input size

92dc8a1

Fix squeezing to make dimensions match during forward pass

72e3f0f

Add pgen to decoder, also consider OOV words

18f1318

Add function to process OOV words to create extended vocab map and in…

c6e552e

…dex map from inputs to extended vocab map IDs. Also computes largest number of OOV words.

Remove TODO items that were already finished

65a9d06

Add coverage to model, TODO remove the debugging statements after con…

c7ab9b0

…firming correct implementation

Improve documentation: add more comments and delete debugging print s…

af0f9e5

…tatements

Improve documentation, change some print statements and add comments

027d33c

When an OOV word is chosen as the next input to decoder, give the emb…

ad317fc

…edding as the UNK word emb

Remove torch debugging print config

91a6a8f

Add boiler-plate imports and logging setup for trainer

54ab2b2

Add imports for argparsing and base model file

b48e1fe

Add trainer init, method to build the model after parsing args, and b…

c05556b

…egin method for training model

Write main to accept args from cli

e1b4c98

Delete documentation comments

82902a1

add argparse and load in vars for training

2b08784

Improve documentation for trainer init

7368b3c

Add default values for train, eval, save, and wordvec pretrain files

99cd01c

Update constants

e3950d8

Add file parsing for train, eval, and wordvec pretrain files. Raise e…

4936bb2

…xceptions if they're missing.

Move model components to device. Add dropout to Encoder after the LST…

a23661a

…M, and after attention layer and the final linear layer in the decoder

Set up training loop with sample data, awaiting proper data pipeline

52c6e22

Fix in-place tensor manipulation bug

a8ee27c

SecroLoL added 29 commits October 7, 2025 16:30

Move model to device

85a959c

Add model checkpoint loading for training

ed5a761

Oops. The checkpoint load file should not be mandatory.

f275f8a

Update logger to be native to decode.py, not passed via evaluate_mode…

668fcb8

…l.py

Fix issues with beam search decoding with None inputs

87d2be2

Add logger statements for debugging evaluation steps

67a594a

Update eval after each training epoch: Use avg loss, not the val set acc

f70f435

Add fixes: use teacher-forcing during training and also use gradient …

ab5faf9

…clipping. print gradient norms

Add layernorm to Encoder, Decoder

533a5bb

Update documentation. Also fix layernorm dims to work with model stru…

035baf8

…cture

Move START and STOP tokens to constants file instead of model file

92a80ab

Put LayerNorm right before sigmoid (correctly now). Add Xavier initia…

c047e47

…lization for linear layers

Clumsy, missing the arg to sigmoid fixed

a9a6686

Alter checkpoint loading to include coverage if not included before

fcc35e4

Since the prev_coverage is a tensor, need to check the elements for N…

6e64b64

…one, not the object itself

Implement validation loss for model selection

f08cd28

Rename helper functions for eval

2e74f05

Fix error: use Softmax instead of LogSoftmax, and apply log at the ve…

99a926b

…ry end of the distribution calculation

Touchups to last modification

7ab8466

Delete old sections of training loop: gradnorm debugging and evaluati…

ef272b7

…on sections

Introduce more dropout to enhance regularization of the model

faca33c

Add weight decay to Adam for L2 reg

6502249

Revert LSTM dropout changes. cannot use built-in dropout with single-…

f40c535

…layer LSTMs

Add verbose option for model eval: prints out reference summaries and…

1083e21

… decoded examples

Bugfix coverage being None when adding extension to hypothesis during…

01f7f22

… beam search

Update truncation for inputs: Add a STOP token to the end of the summ…

fc6ef06

…aries.

remove debugging statement

bd4f686

Bugfix padding tokens not appearing properly and handled by the loss fn

ee9647a

Add error-checking for padding token, and update validation loss to u…

4c19a0f

…se padding too

AngledLuffa closed this Oct 7, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Alex remove lemma rebased #1518

Alex remove lemma rebased #1518

Uh oh!

AngledLuffa commented Oct 7, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Alex remove lemma rebased #1518

Alex remove lemma rebased #1518

Uh oh!

Conversation

AngledLuffa commented Oct 7, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants