Alex remove lemma rebased#1518

Closed

AngledLuffa wants to merge 99 commits intodevfrom

alex_remove_lemma_rebased

Commits on Oct 7, 2025

Begin baseline model; skeleton code

SecroLoL
authored and
AngledLuffa
committed
Initial baseline + pointergen model. TODO fix dimensionalities and introduce p_gen and coverage

SecroLoL
authored and
AngledLuffa
committed
Add baseline model (seq2seq no coverage, no pgen)

SecroLoL
authored and
AngledLuffa
committed
Add constants for model building and args loading

SecroLoL
authored and
AngledLuffa
committed
Delete out logging statements

SecroLoL
authored and
AngledLuffa
committed
Add printing settings to see deeper into tensors

SecroLoL
authored and
AngledLuffa
committed
Add masking to the attention lyaer

SecroLoL
authored and
AngledLuffa
committed
add embedding dim to the decoder linear layer input size

SecroLoL
authored and
AngledLuffa
committed
Fix squeezing to make dimensions match during forward pass

SecroLoL
authored and
AngledLuffa
committed
Add pgen to decoder, also consider OOV words

SecroLoL
authored and
AngledLuffa
committed
Add function to process OOV words to create extended vocab map and index map from inputs to extended vocab map IDs. Also computes largest number of OOV words.

SecroLoL
authored and
AngledLuffa
committed
Remove TODO items that were already finished

SecroLoL
authored and
AngledLuffa
committed
Add coverage to model, TODO remove the debugging statements after confirming correct implementation

SecroLoL
authored and
AngledLuffa
committed
Improve documentation: add more comments and delete debugging print statements

SecroLoL
authored and
AngledLuffa
committed
Improve documentation, change some print statements and add comments

SecroLoL
authored and
AngledLuffa
committed
When an OOV word is chosen as the next input to decoder, give the embedding as the UNK word emb

SecroLoL
authored and
AngledLuffa
committed
Remove torch debugging print config

SecroLoL
authored and
AngledLuffa
committed
Add boiler-plate imports and logging setup for trainer

SecroLoL
authored and
AngledLuffa
committed
Add imports for argparsing and base model file

SecroLoL
authored and
AngledLuffa
committed
Add trainer init, method to build the model after parsing args, and begin method for training model

SecroLoL
authored and
AngledLuffa
committed
Write main to accept args from cli

SecroLoL
authored and
AngledLuffa
committed
Delete documentation comments

SecroLoL
authored and
AngledLuffa
committed
add argparse and load in vars for training

SecroLoL
authored and
AngledLuffa
committed
Improve documentation for trainer init

SecroLoL
authored and
AngledLuffa
committed
Add default values for train, eval, save, and wordvec pretrain files

SecroLoL
authored and
AngledLuffa
committed
Update constants

SecroLoL
authored and
AngledLuffa
committed
Add file parsing for train, eval, and wordvec pretrain files. Raise exceptions if they're missing.

SecroLoL
authored and
AngledLuffa
committed
Move model components to device. Add dropout to Encoder after the LSTM, and after attention layer and the final linear layer in the decoder

SecroLoL
authored and
AngledLuffa
committed
Set up training loop with sample data, awaiting proper data pipeline

SecroLoL
authored and
AngledLuffa
committed
Fix in-place tensor manipulation bug

SecroLoL
authored and
AngledLuffa
committed
Move optimizer zero-ing out the gradient to top of training loop

SecroLoL
authored and
AngledLuffa
committed
Include add_unsaved_module method to allow for char embeddings

SecroLoL
authored and
AngledLuffa
committed
Add CharLM embeddings to model (optionally)

SecroLoL
authored and
AngledLuffa
committed
Add CharLM option to trainer

SecroLoL
authored and
AngledLuffa
committed
Start beam search decoding

SecroLoL
authored and
AngledLuffa
committed
add utils function to convert text to vocab token ids

SecroLoL
authored and
AngledLuffa
committed
Add CharLM to generate embedding for next word in decoding if not using teacher-forcing (Training loop)

SecroLoL
authored and
AngledLuffa
committed
Remove original pointergen dir (accidentally included)

SecroLoL
authored and
AngledLuffa
committed
add helper methods to model class to assist with decoder beam search

SecroLoL
authored and
AngledLuffa
committed
Build beam search algorithm

SecroLoL
authored and
AngledLuffa
committed
Add bugfixes to the summary decoder, using proper inputs. Save model in training loop.

SecroLoL
authored and
AngledLuffa
committed
Add preprocessing to data for loading CNN dataset

SecroLoL
authored and
AngledLuffa
committed
Update output distribution with pgen to have size of the ext vocab map, not the vocab size + max oov words. Leads to indexing error because it undercounts.

SecroLoL
authored and
AngledLuffa
committed
Implement Dataset loader class

SecroLoL
authored and
AngledLuffa
committed
add new dataloading process to trainer loop

SecroLoL
authored and
AngledLuffa
committed
add rouge eval for trained model

SecroLoL
authored and
AngledLuffa
committed
Update default train and eval path roots

SecroLoL
authored and
AngledLuffa
committed
Add argparse

SecroLoL
authored and
AngledLuffa
committed
Add function header documentation

SecroLoL
authored and
AngledLuffa
committed
Add argparse

SecroLoL
authored and
AngledLuffa
committed
Add eval return objects for ROUGE

SecroLoL
authored and
AngledLuffa
committed
add eval to train loop after every epoch

SecroLoL
authored and
AngledLuffa
committed
Add enc_steps and dec_steps limits to truncate inputs if needed. Also, add ability to decode OOV words via ext vocab instead of base vocab

SecroLoL
authored and
AngledLuffa
committed
add max_enc_steps and max_dec_steps. Also, use OOV vocab inclusive mapping.

SecroLoL
authored and
AngledLuffa
committed
add max_enc_steps and max_dec_steps. Also, use OOV vocab inclusive mapping.

SecroLoL
authored and
AngledLuffa
committed
add max_enc_steps and max_dec_steps. Also, use OOV vocab inclusive mapping.

SecroLoL
authored and
AngledLuffa
committed
add max_enc_steps and max_dec_steps. Also, use OOV vocab inclusive mapping.

SecroLoL
authored and
AngledLuffa
committed
Add enc steps dec steps too

SecroLoL
authored and
AngledLuffa
committed
Remove prints

SecroLoL
authored and
AngledLuffa
committed
Add [START] and [STOP] tokens to embeddings layer

SecroLoL
authored and
AngledLuffa
committed
Code cleanup: add tqdm bar to training and log exceptions

SecroLoL
authored and
AngledLuffa
committed
Make log errors type ERROR

SecroLoL
authored and
AngledLuffa
committed
Rename evaluation script due to name overlap with HF library

SecroLoL
authored and
AngledLuffa
committed
Add random sampling filler for batches that are not evenly divisible

SecroLoL
authored and
AngledLuffa
committed
Update trainer

SecroLoL
authored and
AngledLuffa
committed
Add util function to generate checkpoint for model paths

SecroLoL
authored and
AngledLuffa
committed
remove TODO items that are finished

SecroLoL
authored and
AngledLuffa
committed
Change some variable names and improve readability

SecroLoL
authored and
AngledLuffa
committed
Add logging + addtl documentation

SecroLoL
authored and
AngledLuffa
committed
Bugfix the coverage detach None issue

SecroLoL
authored and
AngledLuffa
committed
Move model to device

SecroLoL
authored and
AngledLuffa
committed
Add model checkpoint loading for training

SecroLoL
authored and
AngledLuffa
committed
Oops. The checkpoint load file should not be mandatory.

SecroLoL
authored and
AngledLuffa
committed
Update logger to be native to decode.py, not passed via evaluate_model.py

SecroLoL
authored and
AngledLuffa
committed
Fix issues with beam search decoding with None inputs

SecroLoL
authored and
AngledLuffa
committed
Add logger statements for debugging evaluation steps

SecroLoL
authored and
AngledLuffa
committed
Update eval after each training epoch: Use avg loss, not the val set acc

SecroLoL
authored and
AngledLuffa
committed
Add fixes: use teacher-forcing during training and also use gradient clipping. print gradient norms

SecroLoL
authored and
AngledLuffa
committed
Add layernorm to Encoder, Decoder

SecroLoL
authored and
AngledLuffa
committed
Update documentation. Also fix layernorm dims to work with model structure

SecroLoL
authored and
AngledLuffa
committed
Move START and STOP tokens to constants file instead of model file

SecroLoL
authored and
AngledLuffa
committed
Put LayerNorm right before sigmoid (correctly now). Add Xavier initialization for linear layers

SecroLoL
authored and
AngledLuffa
committed
Clumsy, missing the arg to sigmoid fixed

SecroLoL
authored and
AngledLuffa
committed
Alter checkpoint loading to include coverage if not included before

SecroLoL
authored and
AngledLuffa
committed
Since the prev_coverage is a tensor, need to check the elements for None, not the object itself

SecroLoL
authored and
AngledLuffa
committed
Implement validation loss for model selection

SecroLoL
authored and
AngledLuffa
committed
Rename helper functions for eval

SecroLoL
authored and
AngledLuffa
committed
Fix error: use Softmax instead of LogSoftmax, and apply log at the very end of the distribution calculation

SecroLoL
authored and
AngledLuffa
committed
Touchups to last modification

SecroLoL
authored and
AngledLuffa
committed
Delete old sections of training loop: gradnorm debugging and evaluation sections

SecroLoL
authored and
AngledLuffa
committed
Introduce more dropout to enhance regularization of the model

SecroLoL
authored and
AngledLuffa
committed
Add weight decay to Adam for L2 reg

SecroLoL
authored and
AngledLuffa
committed
Revert LSTM dropout changes. cannot use built-in dropout with single-layer LSTMs

SecroLoL
authored and
AngledLuffa
committed
Add verbose option for model eval: prints out reference summaries and decoded examples

SecroLoL
authored and
AngledLuffa
committed
Bugfix coverage being None when adding extension to hypothesis during beam search

SecroLoL
authored and
AngledLuffa
committed
Update truncation for inputs: Add a STOP token to the end of the summaries.

SecroLoL
authored and
AngledLuffa
committed
remove debugging statement

SecroLoL
authored and
AngledLuffa
committed
Bugfix padding tokens not appearing properly and handled by the loss fn

SecroLoL
authored and
AngledLuffa
committed
Add error-checking for padding token, and update validation loss to use padding too

SecroLoL
authored and
AngledLuffa
committed