Low performance when try to recognize a long string of digits and punctuation

[mouhannadali]

I run the model on the following image:

and I got the following output
[ Word(value='2011CT61-8100/', confidence=0.92),
Word(value='2x2.5+as2.51', confidence=0.41),
Word(value='2011CT61-8200/HDPE', confidence=0.44),
Word(value='50mm', confidence=1.0),
Word(value='2011CT63-801074X2x08#as08', confidence=0.027),
Word(value='2011CTG4-6012X17#os15#os15', confidence=4.9e-06),
........
]

when the model is considering the one string as 2 different strings then the recognition is correct otherwise I am getting the wrong output.

I test the same image on Amazon Textract and the output was correct. any idea what can be done here?

[charlesmindee]

Hi @mouhannadali, Thanks for reporting this.
I guess you used our baseline predictor, which is running a CRNN model for sequence recognition. This model is composed of a 2 LSTM decoder and optimized with a CTC loss, which is less robust on long sequences. This can explain the satisfying results of our model on 2 medium sequences, but the lack of robustness when it has to deal with long sequences.
You may try with attention models (SAR, MASTER) in the recognition predictor, it can lead to better results.
We will focus on lighter attention models in the near future to deal with long sequences, thanks for having reported this. Note that the segmentation seems right because the 2 first lines contain 2 strings separated by a space and the last 4 lines contain 1 long string.
Thank you and have a nice day!

[mouhannadali]

Thanks, @charlesmindee for the fast response. I will try to check if I can find a workaround here to force the model to limit the length of the recognized string to a fixed number of characters. if you know a shortcut or a workaround, I will be more than happy :)

[charlesmindee]

I will think about it within the next days/weeks, one option would be to cut boxes in 2 when more than X chars are detected and redo the prediction on the 2 boxes separately and then merge the 2 char sequences.

[charlesmindee]

Hi @mouhannadali, #465 should fix that!

[mouhannadali]

Perfect, thank you very much. Will check it today :)