SharedPixelBuffer::make_mut_bytes() is quite slow

[leirn]

I am using tiny skia to fill a Image component in slint using the example code proposed here : https://releases.slint.dev/1.7.2/docs/rust/slint/struct.image#

This call :

pixel_buffer.make_mut_bytes();

takes around 2 secondes for a 6500 x 6500 SharedPixelBuffer.

Is there a way to speed up this operation or use another and faster implementation ? Due to lifetime/move issues, I need to call this each time a path is added in my implementation.

[tronical]

Hm, do you need to call this on an existing buffer? I think allocating a new buffer might be faster, especially if you intend to fill the entire buffer anyway.

[leirn]

I tried to cache the SaredPixelBuffer (with very low time improvement), but for lifetime reasons, I need to call pixel_buffer.make_mut_bytes(); each time on the cached SharedPixelBuffer to pass it to a tiny_skia::PixmapMut::from_bytes().

Here what the code looks like :

if self.discard_overlay {
  self.overlay_pixel_buffer = SharedPixelBuffer::<Rgba8Pixel>::new(self.image_width, self.image_height);
}

let data = self.overlay_pixel_buffer.make_mut_bytes();
let mut pixmap = tiny_skia::PixmapMut::from_bytes(data,width,height).unwrap();

I would be happy to put the last to lines inside the if and store them in the struct too, but it gets too complicated (I am not at ease for lifetimes in rust yet, if I can avoid them, I am happy).

[leirn]

Honestly I don't know. My project is in very early stage as I started a couple of days ago as a little experiment, with no knowledge on how Slint or tiny_skia work.

The file I am working on is here if you want to have a look :
https://github.com/leirn/mapannot.rs/blob/main/src/rendering.rs

Basically, the application loads a map (not in the repo even if currently hard-coded in the source) in an image in slint that act as a background.
Another image on top acts as a transparent overlay, and the user can add points, lines, circles and make some measures on them.

The issue I am facing is a 2-second reaction from the tool with large maps when I add an object (a Drawable in the code) to the overlay.

My objective is to reduce this time to a more acceptable one (ideally few ms only) when adding elements.

Not sure my implementation is right and follow the best practices or pattern for this kind of stuff. I don't usually play with GUIs.

[tronical]

Thanks for the code pointer. I tried running it, but I'm indeed missing a file.

From a glance at the code, I can see two functions:

render_background: It seems that this always starts with bg_pixel_buffer and blends layers (with BlendMode::Source) on top. So every time that function is called, it must start with a fresh copy of bg_pixel_buffer. The let mut pixel_buffer = self.bg_pixel_buffer.clone(); call will be fast, but pixel_buffer.make_mut_bytes() is required to re-allocate and make a copy of the entire 6500 x 6500 buffer (> 1GiB memory).

render_overlay: This function seems to be similar (operate on the same dimensions), but re-use the overlay buffer unless discard is true.

With plane dimensions that large (6500 pixels per axis), I suggest to consider breaking that into smaller tiles. In the current design a lot of pixels and touched and copied around (time spent) even though adding an element should affect only few overall pixels, right? If you can break the plan into tiles, you may end up updating only those tiles that really changed (are covered by changed elements).

Oliver implemented a tiled map in Slint in this example.

I'd also consider if you could perhaps let the GPU do the blending, by having the background (that's static), and having a slint::Image for each element on top and place it in Slint. Instead of using tiny skia to do the blending in software, let the GPU do the blending of the Image elements.

[leirn]

Thank you very much for taking time to help me !

You can find the file I am using at this address : http://kelpi.zabro.free.fr/Web/Images/img529.jpg (you will have to rename it).

The background method is called only once at project start, so I am not working on optimizing it for now. I am currently trying to improve the overlay one which is called after several user actions.

Why would the buffer be more than 1 GiB ? 6500 x 6500 x 4 gives 169MB. I agree it is big, but less thant 1GiB.

Tiling is indeed an option, but it seems to increase complexity by a large factor. I would need to tile the background file, but also compute per tile the drawables I add on top.
Currently with an Image inside a ScrollView, I can very easily zoom in and out and, with really a good performance, except when I have to recreate the overlay.

Regarding the solution you propose to let Slint do the blending, wouldn't it require much more buffers ? One per drawable items ? Or am I missing something ?

[tronical]

Ahh, now I can run it. Thanks.

Regarding the size: Oops, you're right :-)

Looking at your code now, I think what I'd do is this:

Keep your background map as it is, in a scrollview indeed. Put that into a slint::Image once, make it the background, then don't ever touch it again :-). For each drawable that you have, besides the Drawable also have a slint::Image that contains the rendering of it. You could squeeze your Vec<Drawable> into slint::VecModel. Then create a slint::MapModel that maps your Rust Drawable struct to a struct that you declare in .slint: export struct OverlayDrawable { x: length, y: length, data: image}, and expose that MapModel to slint. Then in Slint you can write something like this to place the drawables on top of the map: for drawable in all_slint_drawables: Image { x: drawable.x; y: drawable.y: source: drawable.data } - just like the tiles in the Slint maps tiles example, except that instead of tiles covering everything, you place Image elements for each of your drawable.

Unless the entire map is going to be totally filled with millions of drawables, I think this is going to be much faster.

Similarly you could implement layers.

I hope this helps a little. It's a bit tricky to explain this in words - so don't hesitate to ask more :-)

[leirn]

Oh, it looks promising. That's good food for thought.

Thank you very much. I'll work on this in th ecoming days.

[leirn]

You last idea works like a charm. Thank you for your help !