Using docTR predictor with a trained recognition model

[Atomme1]

Hey everyone 🖖

I managed to fully train a recognition model with my own dataset (🎉 yay me 🎉), i did it on a NoteBook in GoogleColab.

After training, the model directory appeared in the files hierarchy with the name « crnn_vgg16_bn_20220602-092736 » because i've trained it using the crnn_vgg16_bn architecture on the 2 of May 2022.

Therefore I wanted to use this new model using DocTR to see if the training was succesfull and i did something out of malice haha.

I have looked where «the crnn_vgg16_bn» model is downloaded on my collab
Downloading https://github.com/mindee/doctr/releases/download/v0.3.0/crnn_vgg16_bn-76b7f2c6.zip to /root/.cache/doctr/models/crnn_vgg16_bn-76b7f2c6.zip

And thus on my PC I have looked for the same path and i litterally delete and swap with my new model.
It worked well 😆 but I wonder if there was an other way to do it, i mean «safer»

Cheers 🥂

[felixdittrich92]

Hi @Atomme1 👋 ,
really nice to hear, that you managed the training :)
For your question i need some more information which doctr version do you use and maybe a code snippet ? ^^

[Atomme1]

I use the latest (1.9.0)
That was the code line i used on my Google Colab
import os
import json
os.chdir ("/content/doctr/")
!python references/recognition/train_tensorflow.py crnn_vgg16_bn --train_path /content/doctr/Train --val_path /content/doctr/Test --epochs 2 --pretrained --vocab legacy_french

After training i download my trained model here called «crnn_vgg16_bn_20220602-092736»

But the thing is when i write these lines to run doctr on a png there is no simple way of using my trained model on the «reco_arch» parameter because it downloads your models instead.

So the thing i did was to look up for the directory of the model it downloads (hidden in my /.cache/doctr/models/ ) and swap the files with my new trained model

from doctr.io import DocumentFile
from doctr.models import ocr_predictor
import json
model = ocr_predictor(det_arch='db_resnet50', reco_arch='crnn_vgg16_bn', pretrained=True)

[dhea1323]

Hey, Is crnn_vgg16_bn architecture possible for training with handwritten words?

[frgfm]

As mentioned by @felixdittrich92, yes it's possible! That might not yield as good results as for typed words though.

We'll start conducting experiments soon about handwritten 👍

[felixdittrich92]

@Atomme1 that's possible with the next release (currently on dev) like:

model = crnn_vgg16_bn(pretrained=False, vocab=VOCABS["french"])
model.load_state_dict('path, mapping='cpu')
predictor = ocr_predictor(reco_arch=model, pretrained=True)

@dhea1323 yes it is but i think the results will be not really good
PS: sry wrote from mobile phone will answer this later a bit more in detail 😅

[Atomme1]

Thanks for the heads up !

Have a wonderfull weekend !

Cheers 🥂 😎