Animated Webp decoder

[young-energy]

Animated webp decoder

It looks like the libwebp cwebp code doesn't support decoding animated webp.
I can't find this feature anywhere which is a bit of a puzzling problem after all this time.

However I noticed in the webpmux muxer code, there is the possibility to extract frames from an animated webp. So it could be possible to use this official sublibrary to effectively decode a webp file.

[jamsinclair]

Thanks @pseudomrando, great request!

It looks like it's not too difficult to implement... I've hacked together a working version in #75.

I've published a special beta version at 1.5.0-animated-webp-support. If you could help confirm it works for you, that would be super helpful.

Usage instructions:

// Firstly run: `npm install -S @jsquash/webp@1.5.0-animated-webp-support`

import { decodeAnimated } from '@jsquash/webp';

const animatedImageBuffer = /* obtain a file buffer of your webp animation */;

// Decode to frames with `decodeAnimated`
// This might take a long time and block the main thread when it is a big image or many frames.
const frames = await decodeAnimated(animatedImageBuffer); 

The frames output will be an array of objects with the type { imageData: ImageData, duration: number }. The duration is how long the frame is shown for in milliseconds before advancing to the next frame (if I understand the library correctly).

Please let me know how you get along with it!

[young-energy]

It's working well on my end! thanks so much!

This is awesome, you've no idea how much I've been pulling my hair trying to find this feature.

This is how I use it in a webpage javascript code:

import * as webpDecode from "@jsquash/webp/decode";  // first run: npm install @jsquash/webp@1.5.0-animated-webp-support
const WEBP_DEC_WASM_LOCATION = "./wasm/webp_dec.wasm"; //make the wasm file available somewherelocation

const wasmFile = await fetch(WEBP_DEC_WASM_LOCATION);
const wasmBuffer = await wasmFile.arrayBuffer();
const wasmModule = await WebAssembly.compile(wasmBuffer);
await webpDecode.init(wasmModule);
const frames = await webpDecode.decodeAnimated(webPBuffer);

My next challenge is to try and make this work in a Cloudflare worker...

Can I rely on @jsquash/webp@1.5.0-animated-webp-support for future automated builds?

[young-energy]

Any advice for it to work in a Cloudflare worker?

I'm doing this

import * as webpDecode from '@jsquash/webp/decode';
import WEBP_FILE from './webp_dec.wasm';

await webpDecode.init(WEBP_FILE);
const frames = await webpDecode.decodeAnimated(webPBuffer);

but the last line is not executed correctly (silent error, so no idea. A console log before displays but a console log right after doesn't)

[jamsinclair]

@pseudomrando, Unfortunately, I don't think animation decoding is suited for Cloudflare Workers, unless you know it will only be tiny images.

The reason is around memory usage. Decoding multiple images can quickly become memory intensive.

Cloudflare workers are restricted to 128MB of memory.

If you had a 1280px by 640px animation that was a short 3 seconds long and had 20 frames per second you would already use over 196MB of memory just storing the RGBA data, excluding other things that are needed to be stored. This would exceed the 128MB restriction and the worker would likely fail every time.

I'm not sure of your use case... but why not run the animation extraction in the browser? You could run it in a web-worker so it doesn't affect the performance of the main web page.

I have Cloudflare examples that hopefully should work if you want to still proceed with it.

[jamsinclair]

Can I rely on @jsquash/webp@1.5.0-animated-webp-support for future automated builds?

Of course! It's published to NPM so it's not going anywhere else anytime soon.

I'll look at eventually adding this properly to the main library releases, but I'll take some time to think through it first.

[young-energy]

I'm working with tiny pixel art animated webp. They have maximum 5 frames.
Here's an example:
https://www.laconiclions.io/LaconicLion_2465.webp
They're 1 or 2 kb each.

I'm trying to create a worker using Cloudflare to convert them as GIFs on demand. As a first step, I was able to use the "decode" function the other day for a single frame webp in the worker, but I don't know how I did it anymore (as I tried so many things).

Now I'm using the following code to decode animate webp thanks to your new function but somehow doesn't work:

import * as webpDecode from '@jsquash/webp/decode';
import WEBP_FILE from './webp_dec.wasm';

await webpDecode.init(WEBP_FILE);
const frames = await webpDecode.decodeAnimated(webPBuffer);

This is basically your Cloudflare suggestion, if I'm not mistaken, I don't know why the last line wouldn't execute tbh.

Could it be that it only works with "decode" and not "decodeAnimated" in this particular setup?

[jamsinclair]

@pseudomrando Hmm have you moved the location of the wasm file? From my knowledge, it'll need to imported from the relative node_modules path.

I've created an example worker below that should generate an animated gif file from an animated webp url.

NPM Dependencies:

• @jsquash/webp@1.5.0-animated-webp-support
• gifski-wasm@2.1.1

Test URLs
Assuming you're using wrangler to run a dev environment

• http://localhost:8787/?url=https://mathiasbynens.be/demo/animated-webp-supported.webp
• http://localhost:8787/?url=data:image/webp;base64,UklGRqgG...

Worker

import * as webpDecode from '@jsquash/webp/decode.js';
import WEBP_DECODE_WASM from '../node_modules/@jsquash/webp/codec/dec/webp_dec.wasm';

import * as gifski from 'gifski-wasm';
import GIFSKI_WASM from '../node_modules/gifski-wasm/pkg/gifski_wasm_bg.wasm';

export default {
  async fetch(request, env, ctx) {
    // Init WASM modules
    const webpInitPromise = webpDecode.init(WEBP_DECODE_WASM);
    const gifskiInitPromise = gifski.init(GIFSKI_WASM);

    // Fetch the Webp image
    const url = new URL(request.url).searchParams.get('url');
    let imageBuffer;

    // Handle data URIs or fetch the image
    if (url.startsWith('data:')) {
      const base64 = url.slice(url.indexOf(',') + 1).replace(/\s/g, '+');
      imageBuffer = Uint8Array.from(atob(base64), (c) => c.charCodeAt(0)).buffer;
    } else {
      const response = await fetch(url);

      if (!response.ok) {
        throw new Error(`Failed to fetch image: ${response.status} ${response.statusText}`);
      }

      imageBuffer = await response.arrayBuffer();
    }

    // Decode the Webp animations
    await webpInitPromise;
    const frames = await webpDecode.decodeAnimated(imageBuffer);

    if (frames.length === 1) {
      // Hack: currently gifski requires at least 2 frames
      // so we duplicate the first frame to create a 2-frame GIF
      frames.push(frames[0]);
    }

    // Create the GIF
    await gifskiInitPromise;
    const gif = await gifski.encode({
      frames: frames.map((frame) => frame.imageData),
      width: frames[0].imageData.width,
      height: frames[0].imageData.height,
      frameDurations: frames.map((frame) => frame.duration),
    });

    // Return the GIF image
    return new Response(gif, {
      headers: {
        'Content-Type': 'image/gif',
        'Access-Control-Allow-Origin': '*',
      },
    });
  },
};

[jamsinclair]

I'm sure you're already aware, but if you do want to use a worker like this in production you should look into caching the gif result.

This should make it a lot more performant and faster for subsequent requests.

See the Cloudflare Worker Cache documentation.

[young-energy]

Thank you so much, this flow works seemlessly!

However I noticed a slight problem in the gif created:

Original: https://www.laconiclions.io/LaconicLion_2465.webp
Converted: https://www.laconiclions.io/LaconicLion_2465.gif

The animation (second frame) occurs after about 10s (which is normal) and only concerns the eyes area. And there's a white (transparent?) artefact instead of the animated area in the created gif.

I'm suspecting something goes unexpectedly in the decodeAnimated(), do you get the same with my webp animation?

[jamsinclair]

Thanks for the report. I know why this is happening. I'll have try to see if we can solve that 🤔

[jamsinclair]

@pseudomrando I have a potential fix and have published a new beta version. Can you try install @jsquash/webp@1.5.0-animated-webp-support-beta.1 and see if that solves it for your images?

Also, if you have time, please play around with a few more animated images and let me know if you spot anything odd. There still might be some edge cases that I am not handling correctly.

[young-energy]

I'm afraid it doesn't. I get the same artifact around the eyes when the 2nd frame renders. Did it work on your end?

[jamsinclair]

Could you remove your node_modules directory and try to re-install with npm install --save @jsquash/webp@1.5.0-animated-webp-support-beta.1? (or your preferred package manager).

It seems to work on my end. I've created a test repo and deployed a web app for quick debugging.

Web App: https://jamsinclair.github.io/webp-animated-test/

When I test the problematic webp image I get this result:

Thanks for helping with debugging this!