Unable to run inference (nor load) a Pretrained NeMo Model

[avishaan]

I'm trying to use the pretrained Megatron style model located here https://catalog.ngc.nvidia.com/orgs/nvidia/teams/clara/models/gatortron_s but am struggling getting up and running. I tried to look through the tutorials around but everything seems to load a model via nemo.collections.nlp.models which it doesn't appear the models in NGC are available in.

I'm familiar with Huggingface as well as building models from scratch via tensorflow but can't figure out how to use this pretrained model. I'm focusing on entity recognition and classification and do not need to do any fine tuning to start.

Thank you in advance for the help

[okuchaiev]

It looks like this checkpoint was trained with Megatron-LM.
You can try using this script to convert SynGatorTron model to NeMo https://docs.nvidia.com/deeplearning/nemo/user-guide/docs/en/stable/nlp/nemo_megatron/mlm_migration.html

[avishaan]

firstly, I appreciate the response so thank you for that

It seems there already is a .nemo file in that repo so I assume it has already been converted appropriately (see below).

Normally, when I load a model from NGC I would do something like

from nemo.collections.tts.models import FastPitchModel
model = FastPitchModel.from_pretrained("tts_en_fastpitch")

which would load https://catalog.ngc.nvidia.com/orgs/nvidia/teams/nemo/models/tts_en_fastpitch

however, when I do something like

from nemo.collections.nlp.models import MegatronGPTPromptLearningModel
model = MegatronGPTPromptLearningModel.from_pretrained("gatortron_s")

it won't load

I've also done things like

import nemo.collections.nlp as nemo_nlp
nlp_models = [model for model in dir(nemo_nlp.models) if model.endswith("Model")]
nlp_models

and

from nemo.collections.nlp.models import MegatronGPTPromptLearningModel
MegatronGPTPromptLearningModel.list_available_models()

expecting to see something related to gatortron with no luck

[avishaan]

okuchaiev, do you are any of your team have thoughts on this? We are thinking of contributing NeMo models moving forward but understanding why the system doesn't work in this case is blocking us