Firefox Crashes on Fullscreen Video with NVIDIA Hardware Acceleration on Wayland

[samuelborn]

Description:
Firefox crashes when attempting to fullscreen a video (e.g., YouTube) after hardware video acceleration (VA-API) has been successfully enabled and confirmed working via nvidia-smi dmon (decoder usage increases). The crash occurs specifically only when entering fullscreen mode.

System Information:

• GPU: NVIDIA GeForce GTX 1650 Mobile / Max-Q
• OS: Arch Linux (using Gnome Wayland)
• Browser: Mozilla Firefox 140.0.2

Steps to Reproduce:

1. Enable hardware video acceleration in Firefox.
2. Open a YouTube video.
3. Start the video.
4. Observe nvidia-smi dmon showing decoder activity (e.g., dec column increases).
5. Attempt to switch the video to fullscreen mode.
   Expected Behavior:
   The video plays in fullscreen mode without Firefox crashing.

Actual Behavior:
Firefox crashes immediately upon entering fullscreen. The following error is observed in the terminal:

libva info: VA-API version 1.22.0
libva info: User environment variable requested driver 'nvidia'
libva info: Trying to open /usr/lib64/dri-nonfree/nvidia_drv_video.so
libva info: Trying to open /usr/lib64/dri-freeworld/nvidia_drv_video.so
libva info: Trying to open /usr/lib64/dri/nvidia_drv_video.so
libva info: Found init function __vaDriverInit_1_0
libva info: va_openDriver() returns 0
[GFX1-]: (gnome) Wayland protocol error: [destroyed object]: error 7: failed to import supplied dmabufs: Could not bind the given EGLImage to a CoglTexture2D

ExceptionHandler::GenerateDump attempting to generate:/home/born/.mozilla/firefox/0f4adum9.default-release/minidumps/698a952d-bc27-712f-182c-14023ccab212.dmp
ExceptionHandler::GenerateDump cloned child 77761
ExceptionHandler::SendContinueSignalToChild sent continue signal to child
ExceptionHandler::WaitForContinueSignal waiting for continue signal...
ExceptionHandler::GenerateDump minidump generation succeeded
Exiting due to channel error.

Key Error Message:
[GFX1-]: (gnome) Wayland protocol error: [destroyed object]: error 7: failed to import supplied dmabufs: Could not bind the given EGLImage to a CoglTexture2D

[elFarto]

I've just reconfirmed this still works on my machine, and it does. Can you confirm which GPU your wayland compositor is using? I'm assuming it's using built-in one, which would cause the failure when attempting to import an exported DMA-BUF.

I'm not sure we'll ever get cross-GPU export/imports working correctly.

[samuelborn]

Actually I am not getting Firefox to work at all at the moment...
I would love to debug this further but I have no idea what is wrong with my current configuration.
Maybe you have any Idea what is currently wrong with my setup.

Configuration steps

• Downloaded libva-nvidia-driver
• Add nvidia-drm.modeset=1 to kernel params
• Set the following Env vars:

export MOZ_DISABLE_RDD_SANDBOX=1
export NVD_BACKEND=direct
export LIBVA_DRIVER_NAME=nvidia

• Set all of those firefox options

media.ffmpeg.vaapi.enabled = true
media.hardware-video-decoding.force-enabled = true
media.rdd-ffmpeg.enabled = true
media.av1.enabled = false
gfx.x11-egl.force-enabled = true
widget.dmabuf.force-enabled = true

• Installed enhanced-h264ify firefox extension

Verification steps

• Validated kernel param is set: cat /sys/module/nvidia_drm/parameters/modeset output: Y
• WebRenderer enabled

• HARDWARE_VIDEO_DECODING force enabled

• Codec seem supported

• MPV hardware acceleration via mpv --hwdec=auto ... works

[danilw]

what your driver version and gnome version?

[samuelborn]

• nvidia-dkms 575.64.03-1
• libva-nvidia-driver 0.0.14-1
• GNOME 48.3

[thesword53]

Your Firefox is running with the Intel UHD Graphics 630, try unsetting the variable LIBVA_DRIVER_NAME to allow VAAPI to automatically choose the right GPU. If you want to use Firefox with the Nvidia GPU and GNOME running on the Intel GPU, launch it with prime-run.

[samuelborn]

So I just commented out the specified env var leaving me like this in .zshenv

export nvd_backend=direct
export moz_disable_rdd_sandbox=1
# export libva_driver_name=nvidia

Afterwards rebooted my system.

Firefox + Youtube still uses the Intel GPU 😦

[thesword53]

Firefox + Youtube still uses the Intel GPU 😦

You should run Firefox with the Nvidia GPU for both renderer and VAAPI (prime-run firefox) because if you run Firefox with the Intel GPU for the renderer and the Nvidia GPU for VAAPI, it will fail when importing dma-bufs from Nvidia GPU to Intel GPU as you show in logs because of Nvidia driver limitation. In your case I suggest you use the Intel GPU for decoding and renderer as it supports the same video codecs as the Nvidia and it will draw less power (https://forums.developer.nvidia.com/t/nvdec-forces-gpu-into-p2-cuda-state-much-higher-power-consumption-than-with-vdpau/55466).

[samuelborn]

prime-run firefox

That worked! Now my graphic card does something when watching videos. Thank you ❤️

To make this change persistent would you recommend adding it to exec line of the firefox.desktop file?

[Obdam]

FWIW: removing the libva_driver_name environment variable + running Firefox with prime-run firefox also worked using Hyprland on Arch Linux.