Size of final model differs from pretraining size

[bermeitinger-b]

I'm fine-tuning the stt_de_quartznet15x5 model from v1.0.1 on my own dataset. The downloaded checkpoint has a size of 68 MB. However, after fine-tuning, my final model checkpoint is around 10 MB.

My training:

python speech_to_text.py \
    +init_from_pretrained_model="stt_de_quartznet15x5" \
    model.train_ds.manifest_filepath="mydataset_train.json" \
    model.train_ds.batch_size=128 \
    +model.train_ds.num_workers=32 \
    model.validation_ds.manifest_filepath="mydataset_val.json" \
    model.validation_ds.batch_size=128 \
    +model.validation_ds.num_workers=8 \
    trainer.gpus=-1 \
    trainer.max_epochs=256 \
    exp_manager.name="my_dataset"

The same happens, when using German CommonVoice, so the same data as the pretraining data.

Which parameters are missing?

[titu1994]

Finetuning should not change model size at all. Could you try using restore_from() on the .Nemo file to check if it loads at all and run inference?

[bermeitinger-b]

I think it loads the "default" config file from conf/config.yaml which is a smaller model (?) and then only restores some of the weights if at all.
To finetune, you have to find the actual YAML file for the pretraining model and give it explicitly with --config-path and --config-name.

Is there a way to find the actual used config file for the pretrained models?

[titu1994]

Each pretrained model had the original config that was used to build it inside of model.cfg - you can reuse that to create another randomly initialized model.

[bermeitinger-b]

Yes but the given script has a different logic than I thought. I thought it loads the model from the 'init_from...' key and not from the default config file.

To fine-tune, I have to load the model manually, export the model's config to yaml and then load it again as parameter for the script which then loads the model's cfg again.

Anyway, a preliminary test showed that the correct model is exported. I was wondering why the training for 256 epochs was done in a few hours. Thanks for pointing me into the correct direction.