Font rendering by fonttools

[mozman]

@mbway Hi! I'm experimenting with the fonttools and figured out how to render TrueType fonts without Matplotlib and the best part is that the result looks like the rendering in BricsCAD with only minor differences - although I have no idea about "kerning" and other advanced font features: yellow in the back the fonttools rendering, blue the BricsCAD rendering

More differences to TrueView 2023:

I have to test if the "homemade" rendering is faster or slower than the rendering of Matplotlib and Matplotlib has still one big advantage: the font-manger locates the TrueType fonts automatically on Windows, Linux and macOS.

The fonttools is a pure Python package and the base install has no dependencies - the kind of libraries I like and could imagine adding as a dependency to the ezdxf core library. This would add reliable text measurement to the core library.

Unless further testing hits some showstoppers and I have a replacement for Matplotlib's font manager, I'd like to replace all font rendering for the drawing add-on with my own font rendering class that generates the text paths for the Matplotlib and PyQt backends and more backends to come, the PyMuPDF package is a good candidate for an accurate and fast PDF backend.

[mbway]

great work! matplotlib doesn't really have the best the best API for working with fonts since obviously it isn't really the point of the library. It would be nice for ezdxf to handle fonts in a backend-agnostic way. I'm sure there are performance tricks that could get it at least equal to matplotlib like caching rendered glyphs (which is just an example, I don't know if that would be beneficial).

[mozman]

Progress report:

The white text are hatches created by ezdxf, the magenta text is the text rendered by BricsCAD.

The results for many fonts are good, but for some fonts not so good:

For 3 fonts the results are wider than the rendering of BricsCAD: LiberationSerif, OpenSans and SimSun (chinese font) and i have no idea why. ChatGPT was very helpful for learning how to use fonttools (really!), but this mystery couldn't be solved by ChatGPT, mostly because it is hard to describe the issue in words.

I don't know if there is a special width factor to apply? The glyphs are almost identical, they are just a tiny bit wider.

The Problem is not kerning, i implemented kerning, just to notice that BricsCAD and TrueView do not apply the kerning tables from the TTF.

But the results are better than by Matplotlib: only the monospaced fonts are good

or PyQT: not even the monospaced fonts are good

I cannot compare these renderings with the BricsCAD rendering, so the rendering of LiberationSerif, OpenSans and SimSun is maybe good.

The bad part are non-latin glyphs: CAD applications do use a replacement font when the code-points are not available in a font, I cannot and will not replicate this, and the rendering of the SimSun font is also not good:

The Matplotlib and PyQT renderings are not better:

Last point is the Matplotlib FontManager replacement, i search in these folders recursively for TTF/TTC files:

Windows:

~/AppData/Local/Microsoft/Windows/Fonts
<SystemRoot>/Fonts

Linux:

/usr/share/fonts
/usr/local/share/fonts
~/.fonts

macOS:

/Library/Fonts/

Any better ideas?

[mbway]

if you want I can see what your test files look like in AutoCAD if you want that to be the authority on what the font rendering should look like?

for linux there is the fc-list command, or searching the locations you mention as well as ~/.local/share/fonts, and maybe ~/.local/share/texmf/fonts should be sufficient but I might have missed more places that fonts can be

[mozman]

Thank you and an update:

It seems that the letter "A" is the better choice than the letter "X" to determine the cap height: all latin-fonts are almost pixel perfect compared to BricsCAD and also TrueView:

The SimSun font is just weired all advance_width values in the hmtx table are the same.

test files:
glyph_test.zip

[mbway]

I rendered the files in AutoCAD and found that it has terrible font rendering, so I would say the state your rendering is currently in is very good.

(rendered to PDF)

[mozman]

The problem of AutoCAD seems to be, that it only recognizes fonts installed in "C:\Windows\Fonts", but Windows seems now to prefer to install fonts in the user folder "~/AppData/Local/Microsoft/Windows/Fonts", therefore most fonts are replaced by the font "TXT.shx".

[mozman]

So far accomplished and pushed to the master branch:

• text rendering replaced by fontTools, fontTools and numpy are now hard dependencies
• moved text rendering from backend classes to the Frontend class
• removed methods: draw_text(), get_font_measurements() and get_text_line_width() from BackendInterface
• removed Matplotlib and PyQt text rendering classes
• moved functionprepare_string_for_rendering() from backend.py to frontend.py
• removed the PillowBackend and command - rendering of filled paths is just bad (triangulation), so text was really ugly

Render time doesn't feel slower than before, there's less caching than before, only the unscaled glyph paths are cached now - there's room for improvement if needed, and the numpy integration hasn't even started yet.

@mbway Do you really need the methods enter_entity() and exit_entity() of the BackendInterface? I don't need the entity information in the CADViewer and highlighting of entities is sometimes annoying (see CADKit example A_000217.dxf).
tl;dr: When recording drawing commands, storing the (virtual) entities really consumes a lot of memory without much benefit.

For the new hpgl2 converter add-on I used the same approach as for the drawing add-on, but this data format needed only polylines and filled polygons and the output formats SVG and PDF needed the bounding box in advance for proper rendering. I wrote a recorder class which records the output of the frontend in compact numpy data types, for which i can get the bounding box very fast and also transformation still works. These recordings can be transformed (scaled, moved, rotated and mirrored) and then played back very quickly on any backend.

As the drawing backend now basically also just uses polylines and filled polygons the same approach could be used. With some additional work it would be possible to play back the recordings on both backend types or unify the backends by converting the Properties classes. Reducing the BackenInterface complexity could simplify the implementation. Of course the recorder doesn't need to record everything (e.g. enter/exit entity keeps all virtual entities of block references in recordings alive) and when playing the recording on a complex backend like the PyQtBackend this information is simply missing.

todo:

• replace text measurement done by matplotlib_font_support
• clipping of VIEWPORT and INSERT entities in the Frontend class, removes set_clipping_path() and the clipping process from backends

[mbway]

I think this is a good approach for the general case because as you mentioned it makes the backends simpler, however this may make things more difficult for my particular use case. That's fine by me so long as workarounds are at least possible.

The file format I am converting to is a custom CAD format instead of an image format. The goal was to use the drawing addon to resolve all the nasty dxf details and write out a simplified structure that could be loaded by my tool and rendered, manipulated and examined.

To achieve this I use enter_entity and exit_entity extensively to rebuild the CAD structure from the sequence of drawing commands. For example when an INSERT is entered I push a value onto a stack in my custom drawing backend and any entities entered afterwards are added under my equivalent of an INSERT entity. When the INSERT is exited I pop the value from my stack.

When I encounter text I store the text and the transform but do not actually render it. It is important for my application that the text content is preserved rather than resolved to paths.

I think I can still accomplish what I need from this new structure by having my backend/converter inherit from both Backend and TextRenderer which should hopefully result in pretty much what I had before.

I understand that you may not want enter_entity and exit_entity to be part of the core functionality of the drawing addon. Perhaps there could be an additional interface such as

class DrawingObserver(abc.ABC):
    @abstractmethod
    def enter_entity(...): ...

    @abstractmethod
    def exit_entity(...): ...

Which, like the TextRenderer my backend/converter could inherit to be able to receive those messages but because enter_entity and exit_entity are no longer available to the Backend it wouldn't interfere with your recorded drawing commands?

[mbway]

Also, instead of removing the pillow backend (which is currently the only pixel-unit backend at the moment I think and therefore may be of use to some people?) perhaps it could use the same technique I proposed above where it inherits from both Backend and TextRenderer with draw_text as a no-op to get the same behaviour as before?

Or for the PillowBackend case you could provide a No-op text renderer that can be swapped in when using backends that can't render text very well.

This would require the Frontend to be able to have a TextRenderer passed in but I think this is a better structure anyway for better configurability.

[mozman]

When you need the enter/exit features they will stay - play back may not work as expected on such backends.

PDF export by the hpgl2 add-on is done via PyMuPDF, which also supports export as PNG and SVG, although the hpgl2 SVG backend is a native exporter. A backend for the drawing add-on will be added soon or the hpgl2 backend will be used via replay function.

Supported file formats:

the Page.get_pixmap() method has a dpi argument and supports a transformation matrix

Example with 300 dpi:

much better than the PillowBackend or the MatplotlibBackend

The PDF output:
Abis.pdf

[mozman]

@mbway I refactored the whole fonts module, removed the _matplotlib_font_support module and the use_matplotlib configuration option and changed the FontFace class to store weight and width (renamed from stretch) as integer values as stored in the TTF file. I hope this didn't break too much of your code, sorry!