fix/discuss recommended workflows

[bertsky]

I am surprised to see the following in our current recommendations:

• Ocropy nlbin instead of one of the Olena algorithms
• slow skimage binarize/denoise processors instead of Olena/Ocropy
• only Tesseract deskewing (no Ocropy)
• region clipping after Ocropy page segmentation (not necessary)
• Ocropy line segmentation after Ocropy page segmentation (redundant)
• line clipping after Ocropy line segmentation (not necessary)
• Tesseract line segmentation without resegmentation or line clipping (to remove bbox overlaps)
• Tesseract vs Calamari recognition (should be exchangeable regardless of workflow up to that point)

EDIT (thanks @jbarth-ubhd for reminding me): also

• region clipping after region deskewing (impossible!)

Do these choices have some empirical grounding (measuring quality and/or performance on GT)?

[jbarth-ubhd]

if olena binarization (other methods tested long before), I would recommend wolf:

("Wiener" snippet is from a larger image. Within 1 method left-to-right: +noise. Lines of same contrast at various levels (always dark on bright in this experiment) )

PS: Default settings applied.

PS2: One could argue "preprocess low-contrast first", but how to do this without knowing noise levels to skip etc... I think this is the primary task for binarization

[jbarth-ubhd]

But in Step 5 cis-ocropy-deskew is mentioned ( but not "recommended" in Step 9 (?) )

[jbarth-ubhd]

Hmm.. we are talking about https://ocr-d.de/en/workflows ?

The "Recommendations" at the end of that page?

[bertsky]

Hmm.. we are talking about https://ocr-d.de/en/workflows ?

The "Recommendations" at the end of that page?

Yes!

But in Step 5 cis-ocropy-deskew is mentioned ( but not "recommended" in Step 9 (?) )

I don't mind that it is not recommended in step 9, because IMHO on the region level orientation is more important than skew, and can differ between region and page, whereas skew is usually uniform across a page (otherwise you usually need dewarping anyway).

The above list really only concerns the overall workflow configuration recommendations.

if olena binarization (other methods tested long before), I would recommend wolf

Thanks again for that test suite and for paying attention in general on that front, which is not appreciated enough IMO. We've briefly discussed this in the chat already, and I would like to elaborate on some open points:

1. The example image is interesting and helpful to see what's going on, but it is also misleading, because it is highly artificial: Algorithms with local thresholding are quite sensitive to highly localized contrast/brightness changes, esp. if they are discontinuous. But realistically, except for the special case of inverted text, these would be spread more widely and continuously across the page. Even more so for noise, which usually appears equally across the page, which is why raw denoising typically measures noise levels globally.
2. Binarization does not (have to) stand alone. If we know contrast/brightness is far from normal, and noise is perceptible, then we would run normalization and raw denoising before anyway. Of course some algorithms are more robust against either of those than others. But if we want a fair competition, we should eliminate them (because we can).
3. Most algorithms have 2 degrees of freedom: the window size (influencing locality; dependent on pixel density) and threshold level (influencing stroke weight). One should allow optimising for them, or at least representing different choices for them. For example, ocrd-skimage-binarize and ocrd-olena-binarize (since v1.1.11) do already set the window size automatically based on a DPI rule of thumb by default. (But this requires having correct DPI annotated.) Niblack is one of those algorithms which is extremely sensitive to the correct window size.
4. As in point 2, but after binarization: Some algorithms produce noise which can be easily removed with a simple binary denoiser. So for a fair comparison all methods should enjoy that benefit.

So I think we need a different artificial test bed. And then we also need a set of true images of various appearances and qualities.

PS2: One could argue "preprocess low-contrast first", but how to do this without knowing noise levels to skip etc... I think this is the primary task for binarization

I disagree. IMO there should be specialisation and modularisation. So binarization processors can concentrate on its core problem, and others can try to solve related ones. If a processor chooses to do 2 or 3 steps in one, fine (we've seen this elsewhere), but we should always have the option to freely combine. And that in turn means we must do it for a fair evaluation, too.

[jbarth-ubhd]

1. The example image is interesting and helpful to see what's going on, but it is also misleading, 
because it is highly artificial: Algorithms with local thresholding are quite sensitive to highly 
localized contrast/brightness changes, esp. if they are discontinuous. 

The word "Wiener" is just a snippet from a 1238 × 388 px image, but I now realize that I forgot to set the DPI.

2. Binarization does not (have to) stand alone. If we know contrast/brightness is far 
from normal, and noise is perceptible, then we would run normalization and raw 
denoising before anyway. Of course some algorithms are more robust against either 
of those than others. But if we want a fair competition, we should eliminate them (because we can

We have images with weak contrast, partly with black surrounding background. Then we must do preprocessing after the clipping step, too, I assume.

Will redo with DPI set.

[jbarth-ubhd]

With imagemagick convert -contrast-stretch 1%x7% # assumption: black on white and 300 DPI:

[bertsky]

The word "Wiener" is just a snippet from a 1238 × 388 px image, but I now realize that I forgot to set the DPI.

Wow! They rarely come in as pristine in quality as this!

We have images with weak contrast, partly with black surrounding background. Then we must do preprocessing after the clipping step, too, I assume.

If you mean cropping instead of clipping, then yes. And contrast/brightness normalization afterwards.

With imagemagick convert -contrast-stretch 1%x7% # assumption: black on white and 300 DPI:

Thanks! Very interesting. You immediately see the class of algorithms that are most sensitive to level dynamics: Sauvola type (including Kim). They are usually implemented without determinining the r from the input. (For example, Olena/Scribo just uses #define SCRIBO_DEFAULT_SAUVOLA_R 128 in a uint8 space.)

But just to be sure: did you use the window size zero default for Olena (which should set it to the odd-valued number closest to DPI)? I would expect Niblack to look somewhat better...

Also, your contrast-stretch recipe is somewhat different to normalize as used by Fred's IM script textcleaner or by one of ocrd_wrap's presets. I wonder what made you consider increasing the white-out tolerance up to 7%. Perhaps we can get to an OCR-specific optimum preset here?

[jbarth-ubhd]

cropping instead of clipping

yes

did you use the window size zero default for Olena

I did not add any parameter -- defaults only.

I wonder what made you consider increasing the white-out tolerance up to 7%

from here: https://sourceforge.net/p/localcontrast/code/ci/default/tree/ctmf.c , line 362.

7/1 = white/black ratio when using "Stempel Garamond" Font (black on white) with reasonable leading. This is what I am used to do when I don't know better. The base idea is that black + white pixels are gaussian distributed and far enough apart, so I crop "equally" on both ends.

[bertsky]

from here: https://sourceforge.net/p/localcontrast/code/ci/default/tree/ctmf.c , line 362.

7/1 = white/black ratio when using "Stempel Garamond" Font (black on white) with reasonable leading. This is what I am used to do when I don't know better. The base idea is that black + white pixels are gaussian distributed and far enough apart, so I crop "equally" on both ends.

Oh I see. So IIUC your logic goes:

• take an average Antiqua-style (modern serif) font,
• look at its white/black ratio when set with an average leading
• assume this will be the minimum white/black ratio, because between text blocks there will be even more white
• expect input to be a binary distribution of the signal (non-normalized) plus some Gaussian noise (but not large enough to make both peaks coincide)
• generally, try to normalize the dynamic range by stretching the histogram to the full dynamic before thresholding
• but prior to that, try to make better use of the dynamic range in the interesting area between both history peaks (pushing them further apart) by clipping the extremes beyond the peaks
• since both peaks must be expected to be of different heights, with at least as much more white than black as in the artificial consideration above, clip 7 times more of the brightest pixels to absolute white than of the darkest pixels to absolute black

So you could have used convert -contrast-stretch 0.5%x3.5% by the same argument, right?

Also, I'd like to verify that ratio point for concrete scans, because on average I assume initials, separators, ornaments and images will increase the share of black. (I'll make a coarse measurement on a real corpus to check.)

[jbarth-ubhd]

Yes, this the idea.

Some interesting binarization with much more equations someone else pointed me to: https://arxiv.org/pdf/2007.07350.pdf

[bertsky]

Some interesting binarization with much more equations someone else pointed me to: https://arxiv.org/pdf/2007.07350.pdf

Yes, we've briefly discussed that in the Lobby. Here is the implementation. Unfortunately does not combine with existing local thresholding algorithms yet.

[bertsky]

So I think we need a different artificial test bed.

As a first step, @jbarth-ubhd could you please change your code to do each point in your matrix (i.e. noise columns, brightness rows) on the full image, and only tile it in the final summary image for visualisation?

[jbarth-ubhd]

Yes, this is was I've done. Process the "full" 1238 × 388 px image (from PDF, 300 DPI, DIN A7) and extract the word "Wiener" for compact comparison.