Inferior quality/filesize ratio compared to libaom from 2022, even at low encoding speeds

[Shnatsel]

I've been trying to use ravif as an AVIF encoding backend in https://github.com/Shnatsel/wondermagick

I've observed that the images encoded with ravif have lower SSIM and/or larger file size than those encoded with `imagemagick. I'm using imagemagick 6.x from Ubuntu 22.04, which uses libaom v3.3.0 for encoding.

I am using quality 50 in both encoders, which seems to produce reasonably close results in terms of SSIM and file size. Occasionally the file does come out smaller than with imagemagick at the cost of SSIM; if I adjust the cavif quality up to match imagemagick's SSIM, the file size increases beyond that of libaom.

Dropping the encoding speed helps decrease the file size, but often it doesn't match the libaom size even with --speed=1 which takes much longer to encode than libaom.

When inspecting the metadata, the most apparent difference between the generated files is that imagemagick produces images with 4:2:0 subsampling while ravif produces 4:4:4 images.

Files produced by imagemagick are also always 8-bit, while cavif will emit 10-bit files for images without an alpha channel. I've only messed around with bit depths a little, but it seems that 10-bit images produce a slightly larger file for an ever so slightly better SSIM.

The same holds for the cavif CLI tool from git on commit 9733893 which uses rav1e v0.8.1

[kornelski]

I've tried to do my own comparison, and here's what I've found so far:

1. AOM allocates bits quite differently than rav1e, which makes it very hard to compare images, because the trade-offs are subjective. For example, AOM may distort edges and colors more, but blurs less. To me less blurring is better, but it's hard to quantify how much one part of an image may be mangled to be worth a rougher texture in another part of the image.

2. -quality 50 in imagemagick is rather low quality. This is contributing to the difficulty, because the judgement isn't "does this codec visibly distort the image?" but rather "which of the many distortions is least problematic?". This is too complicated for objective metrics.

3. Your SSIM tool probably doesn't understand color. It's pretty common for SSIM implementations to use only luma. That's how it's specified in the paper, and has a luminance component in the formula, so it can't be applied naively to Cb/Cr channels. It may be applied to RGB channels, but that usually puts 70% of the weight on the G component that is mostly luma. Even tools that do use YCbCr or Lab may subsample chroma themselves to better simulate color perception. It's really hard to find a tool that supports chroma properly. VMAF has been trained on subsampled videos. I think the latest DSSIM v3 barely does it (DSSIM v1 didn't, despite really trying). Butteraugli does, with a caveat it's designed for barely perceptible distortions.

4. rav1e has some suspicious handling of 4:4:4, and I suspect it puts way too much bits in the color channels: chroma_offset for 4:4:4 looks suspicious xiph/rav1e#3444 This may be exaggerating the differences when you normalize for SSIM between chroma-subsampled images and chroma-overkill images.

[Shnatsel]

Thanks a lot for investigating this and for the detailed explanation!

For SSIM I used the imagemagick 7 implementation, the exact command was magick compare -quiet -metric SSIM "$ORIG_FILE" "$CONVERTED_FILE" difference.png, and I've also double-checked some of the images using this JS implementation to make sure imagemagick isn't wildly off base.

xiph/rav1e#3444 does look odd. I hope that fixing it would result in a better quality/size ratio and thanks to chroma-from-luma prediction we won't need to resort to chroma subsampling at all.