Color Management Overhaul

[betalars]

Okay, I have been a bit frustrated with Color Management in Godot for a while now and I think we need to have a serious talk about it.

The thing that pushed me over the Edge is having taken a look at the incorrect ACES implementation, which is as far as I can tell not only wrong on a fundamental level (this is not really how you are supposed to use ACES) but also uses the incorrect conversion formula for what Godot is using it for (applying a filmic LUT to linear sRGB).

Last but not least there is also the topic of Display HDR.

I have a few subjects to go trough so let's hit them one by one. 😅

[betalars]

ACES

What is ACES?

ACES is a huge standardized Color Space built for interoperability. The reason: Hollywood wants to shoot on whatever sensor in whatever camera and outsource the post processing to any VFX company under the sun so they needed a standardized omega color format: ACES.

ACES can take data from any camera sensor or digital color space, convert it to it's omega space, have everyone work within that space, and then export it to any out space you could want. So supporting ACES in Godot at all is somewhat questionable, although it probably has some merit in real time AR compositing and I will admit the filmic ACES to sRGB Transform has a really nice look to it.

Why is Godot doing it wrong?

As far as I can tell, Godot takes it's internal color space, converts it into ACES using an incorrect transform, converts that back to sRGB resulting in an over-crunched image and then applies any other post processing.

A proper ACES pipeline would make Godot first render into it's own space (linear sRGB if I am not mistaken), convert that space to ACES, then do any post processing and LUTs in that standardized space and then convert the output to whatever space the Monitor demands.

I really think ACES should not be used as a Tonemap the way it is right now, it needs to deeper integrate into the rendering pipeline if you want to actually utilize it.

[betalars]

I have talked to the devs:
Properly supporting ACES is not theisable for Godot Core. It is just too ehavy performance-wise and not made for games. But they are working on re-implementing the tone mapping.

[betalars]

Texture Import

ACES Import

When I tried to import ACES reference images to take a look at the implementation of it, Godot did not properly recognize the format and clearly imported it incorrectly, as seen by the middle image imported using the default import settings:

It shows Godot at the top, with a linear color space and both the original .exr (bottom) and a converted .png converted by blender (top) blender and as a reference the blender interface below.

I tried to fiddle with the import settings but did not manage to make the image look like the reference.

General gripes

I generally would like it a lot if it was easier to know and configure what space is used while importing a texture. I am coming from blender and I am used to image nodes that tell you explicitly what space is used so there never is any ambiguity. I wish this was a lot more clear in godot.

I'm not entirely sure how to implement it, but maybe tagging texture data with their respective space and potentially even doing auto-conversions would be helpful.

[betalars]

Scopes

Having an overlay or panel with different scopes so you can better analyze your post processing would be really nice. Can probably be done in an AddOn somehow, but I think it can probably be done a bit cleaner when it is built-in.

[ArchercatNEO]

HDR Output

I saw you mention this but didn't see you say much? It is still a work in progress but I am working on implementing it for Wayland based on an implementation for Windows and all the RenderingDrivers.

The implementation at the RenderingDriver level (for all drivers) and for Windows is happening at godotengine/godot#94496. As of now it seems only HDR10 and scRGB are planned/implemented but in the future there may be additional formats in future PRs. Here there is also talk of ACES 2 if you have suggestions/changes it may be helpful to be involved with how ACES 2 is implemented. The rendering team seems to believe new tonemappers will be required so maybe we can make sure they are implemeneted "correctly" (or incorrect implementations sufficiently justified).

As the one working on the Wayland impl (and not much on the color theory part), I believe future, future work could involve supporting not just colorspaces, but arbitratry primaries and white points as under Wayland this information is actually accessible and the theoretical most efficient apps would make use of these arbitrary primaries.

[betalars]

I was planning on doing more research about that first, so thank you for the input.

[ArchercatNEO]

@betalars just in case you haven't noticed there is now an official issue for wide color gamut support here #12783 maybe you have some feedback/comments on the proposed implementation?