Surface::get_current_texture does not block when the window is fully invisible (macOS)

[ComfyFluffy]

Description
I am currently following a WGPU Tutorial and have implemented Hello Triangle with vertex buffers.

When I click the green button to expand the window into fullscreen, the process will suddenly take up to 22GB of memory:

After a few seconds, the memory usage goes down:

Repro steps
I have uploaded the code onto GitHub.

git clone https://github.com/ComfyFluffy/wgpu-tutorial.git
cd wgpu-tutorial
RUST_LOG=info MTL_HUD_ENABLED=1 cargo run

Then expand the app to fullscreen.

If expanding to fullscreen does not cause memory peak, switching to other desktop then switch back to the app can also cause it.

Resizing on windowed mode does not trigger the behavior.

Resizing on Windows 11 does not trigger the behavior.

Expected vs observed behavior

Expected: The memory usage should remain largely the same during resizing.
Observed: The memory usage goes up to 22 GB.

Extra materials
Info log logged by env_logger. Memory peak happened during resizing.

[2023-11-26T06:43:52Z INFO  wgpu_core::instance] Adapter Metal AdapterInfo { name: "Apple M1 Pro", vendor: 0, device: 0, device_type: IntegratedGpu, driver: "", driver_info: "", backend: Metal }
[2023-11-26T06:43:52Z INFO  wgpu_core::device::global] configuring surface with SurfaceConfiguration { usage: TextureUsages(RENDER_ATTACHMENT), format: Bgra8UnormSrgb, width: 1600, height: 1200, present_mode: Fifo, alpha_mode: Opaque, view_formats: [] }
[2023-11-26T06:43:52Z INFO  wgpu_hal::metal::surface] build swapchain SurfaceConfiguration { swap_chain_size: 3, present_mode: Fifo, composite_alpha_mode: Opaque, format: Bgra8UnormSrgb, extent: Extent3d { width: 1600, height: 1200, depth_or_array_layers: 1 }, usage: TextureUses(COLOR_TARGET), view_formats: [] }
[2023-11-26T06:43:52Z INFO  wgpu_core::device::global] configuring surface with SurfaceConfiguration { usage: TextureUsages(RENDER_ATTACHMENT), format: Bgra8UnormSrgb, width: 1600, height: 1200, present_mode: AutoVsync, alpha_mode: Opaque, view_formats: [] }
[2023-11-26T06:43:52Z INFO  wgpu_core::device::global] Automatically choosing presentation mode by rule AutoVsync. Chose Fifo
[2023-11-26T06:43:52Z INFO  wgpu_hal::metal::surface] build swapchain SurfaceConfiguration { swap_chain_size: 3, present_mode: Fifo, composite_alpha_mode: Opaque, format: Bgra8UnormSrgb, extent: Extent3d { width: 1600, height: 1200, depth_or_array_layers: 1 }, usage: TextureUses(COLOR_TARGET), view_formats: [] }
[2023-11-26T06:43:53Z INFO  wgpu_core::device::global] configuring surface with SurfaceConfiguration { usage: TextureUsages(RENDER_ATTACHMENT), format: Bgra8UnormSrgb, width: 1600, height: 1200, present_mode: AutoVsync, alpha_mode: Opaque, view_formats: [] }
[2023-11-26T06:43:53Z INFO  wgpu_core::device::global] Automatically choosing presentation mode by rule AutoVsync. Chose Fifo
[2023-11-26T06:43:53Z INFO  wgpu_hal::metal::surface] build swapchain SurfaceConfiguration { swap_chain_size: 3, present_mode: Fifo, composite_alpha_mode: Opaque, format: Bgra8UnormSrgb, extent: Extent3d { width: 1600, height: 1200, depth_or_array_layers: 1 }, usage: TextureUses(COLOR_TARGET), view_formats: [] }
[2023-11-26T06:43:56Z INFO  wgpu_core::device::global] configuring surface with SurfaceConfiguration { usage: TextureUsages(RENDER_ATTACHMENT), format: Bgra8UnormSrgb, width: 1600, height: 1256, present_mode: AutoVsync, alpha_mode: Opaque, view_formats: [] }
[2023-11-26T06:43:56Z INFO  wgpu_core::device::global] Automatically choosing presentation mode by rule AutoVsync. Chose Fifo
[2023-11-26T06:43:56Z INFO  wgpu_hal::metal::surface] build swapchain SurfaceConfiguration { swap_chain_size: 3, present_mode: Fifo, composite_alpha_mode: Opaque, format: Bgra8UnormSrgb, extent: Extent3d { width: 1600, height: 1256, depth_or_array_layers: 1 }, usage: TextureUses(COLOR_TARGET), view_formats: [] }
[2023-11-26T06:43:56Z INFO  wgpu_core::device::global] configuring surface with SurfaceConfiguration { usage: TextureUsages(RENDER_ATTACHMENT), format: Bgra8UnormSrgb, width: 5120, height: 2880, present_mode: AutoVsync, alpha_mode: Opaque, view_formats: [] }
[2023-11-26T06:43:56Z INFO  wgpu_core::device::global] Automatically choosing presentation mode by rule AutoVsync. Chose Fifo
[2023-11-26T06:43:56Z INFO  wgpu_hal::metal::surface] build swapchain SurfaceConfiguration { swap_chain_size: 3, present_mode: Fifo, composite_alpha_mode: Opaque, format: Bgra8UnormSrgb, extent: Extent3d { width: 5120, height: 2880, depth_or_array_layers: 1 }, usage: TextureUses(COLOR_TARGET), view_formats: [] }

Platform
Macbook Pro 14 (2021), M1 Pro with macOS Sonoma, 16 GB Memory
wgpu: 0.18.0

[ComfyFluffy]

Setting to fullscreen with window.set_fullscreen(Some(winit::window::Fullscreen::Borderless(None))) (winit window) on macOS will also cause the issue, but not on Windows 11.

The issue does not present when using Vulkan with MoltenVK.

[cwfitzgerald]

Try again now that #4781 has landed

[cwfitzgerald]

This doesn't reproduce for me basically identical setup on trunk. Going to close, if you tell me it's still happeneing I'll reopen.

[ComfyFluffy]

This still happens to me after updating wgpu (2964eed). Using macOS 14.1.1.

Sometimes resizing to fullscreen only will not trigger the issue. However, the issue will present swapping to other desktop using trackpad and then swapping back to the fullscreen app.

I have updated my git repo of the demo.

[ComfyFluffy]

With further investigation using Instrument I found that VM: IOSurface has 9 GB in this run, which traces into CA::SurfaceUtil::CAIOSurfaceCreate(unsigned int, unsigned int, unsigned int, unsigned int, unsigned int, unsigned int, unsigned long long, CA::SurfaceUtil::SurfaceAlignment, __CFString const*).

The stack trace in a single call:

[ComfyFluffy]

With further investigation I located the issue to the execution time of surface.get_current_texture().

In the event loop, I use the following structure to render:

match event {
    Event::WindowEvent { event, .. } => match event {
        WindowEvent::RedrawRequested => {
            match state.render() {
                Ok(_) => {}
                Err(wgpu::SurfaceError::Lost) => {
                    warn!("Lost surface");
                    state.resize(state.size)
                }
                Err(wgpu::SurfaceError::OutOfMemory) => panic!("Out of memory"),
                Err(e) => eprintln!("{:?}", e),
            }
            println!(
                "Rendered in {}ms",
                render_start.elapsed().as_micros() as f32 / 1000.0
            );
        }
        _ => {}
    },
    Event::AboutToWait => {
        state.window().request_redraw();
    }
    _ => {}
}

Generally, state.render() takes about 8ms to finish (on 120hz monitor). Inside render() I use surface.get_current_texture() to get output texture. Normally it takes about 8ms to get the texture before rendering:

However, when putting the window into another desktop, or setting fullscreen mode then switch to other desktops, the window becomes fully invisible. This cause the time of calling surface.get_current_texture() changes to only a few microseconds:

Since the render has finished, Event::AboutToWait triggers, causing it to render again. This results too much frames being draw in a short period of time. Each get_current_texture() call allocates about 22M of memory, overwhelming the system memory.

[Nnubes256]

For those wondering how to mitigate this issue if affected by it while a proper fix lands, this comment I made for the previous big memory leak may help: sotrh/learn-wgpu#207 (comment).

The key thing of these two mitigations is avoiding drawing or limiting the number of frames drawn when the window is occluded, which helped with the previous issue; in this case, it might help in this case if you can detect whether the window is unfocused (thus, in particular, the CVDisplayLink hack I made may fix the issue entirely). Bevy mitigated this, for example, by limiting the rate of frames rendered when the app was in the background, see bevyengine/bevy#7611.