MatchBoxNet for own data and labels

[monologue110]

I'm wondering that if the MatchBoxNet is open for custom data by modifying the configuration file and then training.
I know there are some pre-trained models but I guess not work for the new labels

[titu1994]

Yes you can edit the labels in the config and manifest you create to suite any type of classification for matchboxnet

[monologue110]

A follow-up question:
After training with new labels, I got a pretty good validation result in AUC, loss, and accuracy. The testing result can also achieve a high topk accuracy and low loss. The data is balanced.
However, during the inference, the model is likely to classify the input wave file into the same categories (the output logit is highest in the same label).

Here's the sample code:

wf = wave.open(filename, 'rb')
data = wf.readframes(CHUNK_SIZE)
signal = np.frombuffer(data, dtype=np.int16)
data_layer.set_signal(signal)
batch = next(iter(data_loader))
audio_signal, audio_signal_len = batch
logits = model(input_signal=audio_signal, input_signal_length=audio_signal_len)

[titu1994]

You can check whether the model weights are being properly loaded, and if the number of output classes matches the new label set. If that is the case, then potentially the audio is not being analysed properly - try using model.transcribe(list of audio file paths) and see if it works properly. Also check if model is set to eval mode

[monologue110]

Thanks! Realized that not switch to the eval() model. And now works fine.

Another quick question, during the experiment, I added a new preprocessor class to the audio_preprocessing.py and train based on that. The training and validation loss does not decrease as expected and the results were bad compared the NeMo's preprocessor. Are there any ideas for improvement?

[monologue110]

Sorry, I have another follow-up question that the results are slightly different when running model(input_signal, length) as a whole and step-by-step

model.preprocessor()
model.encoder()
model.decoder()

The output logits value is not the same. Is there anything I missed?