How to save the model trained on FARMReader to Transformer model and upload it on Transformer Hub?

[karndeepsingh]

Hi,
I have trained a Q&A Model using FARMReader and now I have converted it into transformer model using transformer converter which returns prediction head. But I want to upload this trained model on to transformer hub so, I can achieve that. I am unable to save respective config files that are required when converter converts from FARM model to Transformer model.

Please help.

Thanks

[julian-risch]

Hi @karndeepsingh could you please describe in a bit more detail what you've tried and at what point it doesn't work for you? Am I correct that you have used Haystack's Converter class and its functionalities in the following way?

from haystack.modeling.conversion.transformers import Converter
transformers_models = Converter.convert_to_transformers(reader.inferencer.model)

[karndeepsingh]

@julian-risch Thanks for answering. That's the same step I followed to convert it into a transformer model. How to save transformer model into the model file as the whole model got converted and stored in a variable. But I think there are a lot of other things needed like config files and token files etc. How that can be prepared and saved? Because I want to use the trained model with the transformer library, so how I can save the necessary files to use it in the transformer pipeline.

[julian-risch]

Hi @karndeepsingh here are two different ways to save a loaded model. In the screenshot, you can see what files are stored. I hope that helps. 👍

from haystack.nodes import FARMReader
reader = FARMReader(model_name_or_path="bert-base-uncased", use_gpu=True)
reader.save(directory="my_model")

from haystack.modeling.conversion.transformers import Converter
transformers_models = Converter.convert_to_transformers(reader.inferencer.model)
transformers_models[0].save_pretrained(save_directory="my_model_transformers")

[karndeepsingh]

Hi @karndeepsingh, we are currently working on creating a guide on how to deploy haystack pipelines on AWS / Kubernetes. You can follow the progress in this epic. To also be able to view the associated issues to this epic, the Zenhub extension needs to be installed.

Once the model got trained using FARMReader does it saves best checkpoint or last checkpoint? If it saves last checkpoint then how we can save best checkpoint ?

thanks

[bogdankostic]

It saves the last checkpoint, however, in the train method, you can set the parameters checkpoint_every and checkpoints_to_keep to also save previous checkpoints.

[karndeepsingh]

It saves the last checkpoint, however, in the train method, you can set the parameters checkpoint_every and checkpoints_to_keep to also save previous checkpoints.

I can save every checkpoints but which one is best how we get to know?

[tstadel]

@karndeepsingh you can use Trainer's early_stopping param together with the EarlyStopping class. By using early stopping only the best model will be saved.

[karndeepsingh]

@karndeepsingh you can use Trainer's early_stopping param together with the EarlyStopping class. By using early stopping only the best model will be saved.

@tstadel Thanks for answering.
Can you please refer to the document referring to early stopping used as a parameter in Trainer? As, I can't see any parameter related to early stopping in the training method. Following screenshot is for referance: