Feedback on glTF color explainer

[elalish]

I recently wrote up an explainer on what color accuracy means in PBR and glTF, how to think about it, and how to create and present 3D models that accurately represent real-world objects: https://modelviewer.dev/examples/color.html. Of course this is a huge topic, so not nearly everything is covered there. In fact, probably a whole other topic just on tone mapping functions is warranted, though if you're interested, this is a fantastic resource.

I'd like to open a forum here for discussion, as I'd like to know what is clear, what could be improved, and what other related topics the community is interested in (especially if someone would like to contribute their knowledge in another article). Thanks!

[sobotka]

https://app.ruttl.com/share/cjIVHRoJ95X5HC6hH9am?page=J1zfBmfGzLdHVSTCCMRz

[elalish]

Thanks! I think a lot of your comments come down to RGB is a big approximation of true specular transport (there aren't three wavelengths, but a spectrum). Totally valid, but I'd say out of scope for what we care about here. If you happen to have a spectral renderer, it would be great to add to our fidelity tests. I'd be very curious to see exactly how much difference it makes. Certainly a lot in pathological situations, but I wonder how it compares to the various real-time approximation errors in terms of magnitude for these product-style setups (yes, I mean qualitatively perceptual, but hopefully we can agree when we look at the pictures).

[sobotka]

I would suggest that this glazes over a structural problem.

We define our mixtures using tristimulus chromaticities. But those are relative values. The problem is that for any given model surface, we are utilizing those definitions as reflectances.

That means that the quantities of light-proxy reflected varies greatly in terms of tristimulus RGB colourspace to tristimulus RGB colourspace.

There is simply no real method to achieve the purpose statement.

[elalish]

Agreed, but I would argue we have yet more fundamental structural problems than that even. Have you looked at my code for shadows? Horribly distant from physics, yet still does a decent job of convincing a user. I'll only accept that an approximation should be prioritized for improvement if we have example images demonstrating it's causing the biggest problems.

You're right though that feeding a glTF into a spectral renderer is pointless, since it's an RGB format. I think better would be to make a beautiful example model in a spectral format, render that in the spectral renderer, then convert it as accurately as we can to RGB glTF and render in ours for comparison. Do example models like this exist somewhere?

[sobotka]

We can emulate a visible range of spectral light transport quantities. The question I return to is “What should this be manifested as in terms of ‘image’?”

It might seem seductive to say “As though we were standing there looking at it” but this is nothing more than a contemporary mirage; the history of photography from The Cabinet of Dr. Caligari onward would suggest that the image occupies a unique place as artifact, rather than simulacrum. After all, there is literally no filter that can emulate the many different formations behind creative black and white film, for example. Even if the goal were pure simulacrum, such as is the case for perhaps an ECommerce example (although a hundred years of advertising would beg to differ) no such existing models exist.

So this leads to looking specifically at our contexts. If we consider the idea that the colourimetric definition of the tristimulus open domain values represent a specific chromaticity, we are faced with expressing that chromaticity in the medium facing us. Here the open domain will hit limits against the closed domain medium.

As a few of us who have attempted this will tell you, even if we succeed in such an endeavour, and attenuate the chroma of the mixtures to maintain a sense of the source chromaticity “angle” where they cannot be expressed in the medium, it will yield a less than optimal, or even acceptable in many cases, image. This is extremely well covered in MacAdam’s “The Quality of Color Reproduction” (1951). This document should be mandatory reading in engaging the subject at hand.

Specifically, if we were to follow a “by the book” effort to replicate chromaticity at the display, we will find:

1. Appearance notions of hue will disconnect. See also Abney effect.
2. Potential uncanny effects such as the Helmholtz-Kohlrausch effect.
3. The general progression of chroma will be lacking or distorted; film, for example, via the density mechanic, would start at a lower chroma down low, bowl outward and increase chroma, and then taper chroma toward the higher image values.

The point in all of these sorts of discourses is to appreciate that there simply is no scientific solution here, and that all solutions are ultimate an aesthetic of convention.

Even our perceptual systems themselves do not reveal anything about the “reality”, but rather use what we term “colour” as a salient infographic representation of useful features to us as organisms.

We are likely starting on a better foot to accept that there is no absolute ground truth here, and instead work toward identifying a consistency of representation of a formed image at a specific medium. That’s enough work unto itself!

[elalish]

I just found your wonderful blog, but you left us hanging! I certainly hope you find time to write more.