Proposal: Transfer OCR generation to Robotoff

[raphael0202]

Every time an image is uploaded on Open Food Facts, Robotoff is (to my knowledge) the main user of OCR files.
Robotoff uses OCR files to:

• extract product weight, brand name, stores, labels,...
• determine the languages present on the images, the image rotation
• detect abusive images
• predict categories (text detected on images is one of the model input)
• extract nutrition information

Product Opener sends an event on Redis every time an image is uploaded. However, at the time Robotoff processes the event, the OCR file may not be available yet. OCR processing is indeed processed in the background using icron on the server where Product Opener is running.

It happens quite often: in the last 90 days, the insight generation from OCR failed 23k times due to missing OCR file, and ingredient extraction failed 6.4k times.

Because of this, we miss insights for some products.

Option 1: Transfer ownership of OCR generation to Robotoff

If Robotoff generates OCR files instead of Product Opener, we can run prediction tasks that require OCR files after the OCR processing.

Benefits

• we reduce (slighly) CPU and bandwidth load on off2 server
• it's more robust than relying on background processing on off2. If the icron fails silently, no OCR is generated. It happened frequently in the past, much less often now.
• if Google Cloud Vision fails, it's much easier to relaunch it later from Robotoff side.
• We can easily relaunch all insight predictions using OCR files stored on disk.

Shortcomings

• the OCR JSON URLs are no longer valid (ex: https://images.openfoodfacts.org/images/products/406/826/203/8564/2.json or https://static.openfoodfacts.org/images/products/406/826/203/8564/2.json). We can perform a redirect to Robotoff domain so that we don't break existing OCR URLs.

Option 2 : publish ocr_generated event on a new redis stream

Create a new Redis event, on a new stream, when the OCR is ready. Robotoff can listen to this stream and launch jobs that require OCR when it receives the event.

Benefits

• simplicity of implementation

I'm personally in favor of option 1.

[stephanegigandet]

There is also on demand OCR generation (for selected ingredients images) currently used by the website and mobile app. We probably should keep it as-is for the time being (and possibly create a new API on Robotoff that we can move to gradually).

[raphael0202]

I was not included in my proposal the on-demand OCR generation, only the OCR generation that is automatically launched for every new image (using icron).

[alexgarel]

@raphael0202 what about creating an event for "OCR ready" ? So that robotoff can listen to it instead of "image uploaded" ?

Maybe robotoff would have different task to launch on each event ?

FYI it's no more based upon incron, but still it's based on inotify

[raphael0202]

what about creating an event for "OCR ready" ? So that robotoff can listen to it instead of "image uploaded" ?

I suggested this in option 2 ("Option 2 : publish ocr_generated event on a new redis stream")

FYI it's no more based upon incron, but still it's based on inotify

Thank you for the info!

[raphael0202]

In Product Opener call, we decided to go for the new redis stream for now. We can move the feature to Robotoff (or another service) later if needed.

Issue: #12256