Handling of alpha in derived image items

[maryla-uc]

I believe that currently neither HEIF, MIAF nor AVIF specify how to handle auxiliary items when processing derived image items.

For example, when processing a 'sato' (Sample Transformed Derived Image Item), if the input image items have associated alpha, should the alpha be processed too? (I think so, since this is what libavif is doing). In that case, should the spec require that either all inputs have alpha or none of them do, with the same premultiplication?

[leo-barnes]

@maryla-uc, @y-guyon
Is this something we should try to address before publishing v1.2?

[y-guyon]

I believe that currently neither HEIF, MIAF nor AVIF specify how to handle auxiliary items when processing derived image items.

This is partially defined in the draft of ISO/IEC 23008-12 4th edition.

For example, when processing a 'sato' (Sample Transformed Derived Image Item), if the input image items have associated alpha, should the alpha be processed too? (I think so, since this is what libavif is doing). In that case, should the spec require that either all inputs have alpha or none of them do, with the same premultiplication?

https://aomediacodec.github.io/av1-avif/#sample-transform says "the value of a pixel for a given channel". Assuming alpha is part of a pixel, I would answer yes. For depth I would answer no (depth is not part of a pixel). This is also aligned with the suggested changes in ISO/IEC 23008-12 4th edition.

Is this something we should try to address before publishing v1.2?

This may be tricky. v1.2 is quite late stage already, and rushing some additional rules may introduce more edge cases than it removes. Do you have a wording in mind?