Resize an image on the fly?

[dave-yotta]

Looks like Image.Load loads up the whole stream.

I'm looking for a way to resize a large image (to a much smaller image) keeping a smaller buffer, without any seeking.

I could probably devise something that has a limited quality by skipping most of the pixels depending on the stride in the pixel format; would rather use a library to do this - is it something I can do with ImageSharp?

[br3aker]

If you are using JPEG images - yes, ImageSharp now supports really fast downscaled jpeg decoding with scalings 1/2, 1/4 and 1/8 from #2076.

[br3aker]

if it supports a max subsampling of 1/8 then it's more likely that we'll end up using imagesize/8 amount of memory to resize a single image

No, result image after initial built-in scaling would be of size (width / 8) * (height / 8) or width * height / 64. So it's ~64 times less memory allocated for an image.

Jpeg images' size is contrained to 65535×65535 pixels by the specs, it's physically impossible to save image larger than that using JPEG format.

Decoded black and white (1 byte per pixel) image with size 65535×65535 would allocate 4.294.836.225 bytes or 4 gigabytes. This exact image scaled with built-in jpeg scaling would be of size 8192x8192 and would allocate 67.108.864 bytes or 64 megabytes. I assume image processing services not to be bottlenecked by a 500mb RAM hardware :)

Things get a bit trickier with RGB images as they would allocate up to 12 gigabytes unscaled and up to 192 megabytes prescaled. IMO this is still acceptable even for personal computers, not even talking about servers.

Regarding internal buffer for actual decoding: worst case scenario won't allocate more than 20 megabytes on top of it regardless if you'd use built-in scaling or not.

[dave-yotta]

I can just reject them based on the file size which will be a bit easier - sorry you're right of course, 64 times.

If we only supported JPEG that would likely be a non-issue, especially given you're using shared buffers. But we support any image format so we're still at-risk. The image processing runs with the rest of the APIs, which have 2gb available iirc.

Note: I still prefer the idea of stream processing because it can minimize the response time and buffer sizes. We've got plans to show these previews in a grid-style layout on clients (including non-images although it doesn't apply here), which worries me within this context - but we'll likely store reduced size images processed in the background in the end so this problem does go away.

[br3aker]

Thing is, JPEG format allows us to perform in-place scaling out of the box. Other supported image formats do not have such performance opportunities out of the box, they still can be downscaled inplace - it's just harder to implement, a lot of internal codec code must be rewritten to support this.

Stream processing can help a lot in memory consumption but ImageSharp just doesn't work like that.
Current Decode -> Image -> Mutate (optional) -> Encode pipeline just won't work with stream'ed image processing. Maybe one day we'd see such API, maybe not. I'd say an issue can be opened to have an open discussion about this idea with the core team :)

[JimBobSquarePants]

The problem with stream processing as I see it is that it makes things super complicated.

We'd have to develop methods to lazily pull the required (different per mutation operation) scanlines from the encoded stream which means we'd either have to copy or lock the stream. You then might have to do some IO thrashing depending on the position of metadata in the stream relative to that data. We'd have to rewrite every single existing operation to handle a sliding window.

Writing the library itself is hard enough as it is with the very limited number of contributors. You'd be better of buying more RAM.

[dave-yotta]

Completely understand - it's hard enough to implement one resize operation let alone all the mutations you have here if you have access to (e.g.) only one pixel at a time only once, a limited working buffer and an output stream - becomes a totally different algorithm.