Not able to get smooth video playback on raspberry pi 4

[Sravdar]

Is this package ever tested on raspberry pi 4?
What i am trying to achive is playing 1080p and 60 fps video on raspberry pi with few widgets on top of it. (esentially video as background). I have seen some raspberry pi related code in the package so i gave it a shot.

1080p 30 fps video is laging even without widgets on top of it. 720p 30 fps lags when there is animated widgets on top. I am trying to achive hardware acceleration since pi4 has support for it but looking at cpu usage i think it falls back to software decoding.

I would like to learn details if this package has ben run on rasberry pi 4 sucessfully. Or i can use some gudiance for debuging this issue.

fvp: 0.33.0

Package: wayfire
Version: 0.7.5-1~bpo11+1~rpt35

[✓] Flutter (Channel stable, 3.32.4, on Debian GNU/Linux 12 (bookworm) 6.12.25+rpt-rpi-v8, locale en_GB.UTF-8) [242ms]
    • Flutter version 3.32.4 on channel stable at /home/pi/development/flutter
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision 6fba2447e9 (4 weeks ago), 2025-06-12 19:03:56 -0700
    • Engine revision 8cd19e509d
    • Dart version 3.8.1
    • DevTools version 2.45.1

[✓] Linux toolchain - develop for Linux desktop [969ms]
    • Debian clang version 14.0.6
    • cmake version 3.25.1
    • ninja version 1.11.1
    • pkg-config version 1.8.1
    • GL_EXT_framebuffer_blit: no
    • GL_EXT_texture_format_BGRA8888: no

[✓] Connected device (1 available) [239ms]
    • Linux (desktop) • linux • linux-arm64 • Debian GNU/Linux 12 (bookworm) 6.12.25+rpt-rpi-v8

[✓] Network resources [862ms]
    • All expected network resources are available.

[wang-bin]

delete libffmpeg.so.* in your app bundle, then system ffmpeg libraries(hw decoders enabled) will be used.

[wang-bin]

https://github.com/wang-bin/fvp?tab=readme-ov-file#enable-hardware-decoders-for-embedded-linux

[Sravdar]

@wang-bin I have test what you have suggested. Deleting libffmpeg indeed helps with hardware acceleration but it brings other problems.

When app is compiled as linux app on raspberry pi and run on wayland window manager it shows some artifacts (? green bar on bottom). While perofrmance is better on 1080@60 it still have observable performance drop. AV1 videos don't play even though ffplay can play it (not sure if this is normal behaviour). h264 videos freezes the app after replaying few time in random occasions.

When app is compiled with sony/elinux --target-backend-type=gbm and run on dri/card0 performance on h264 and h265 videos are quite good. But they still causes app to freeze after a while. AV1 videos still doesn't play.

May i ask how have you setup raspberry pi?

• What OS?
• Version of ffmpeg (or commit if its compiled from source)
• Which method is used for running the app? Linux build on wayland window manager or elinux?
• Version of flutter

[wang-bin]

But they still causes app to freeze after a while.

Play a video without user interaction and then freeze? Can you show me the log? And can you share a minimal example to reproduce it?

AV1 videos still doesn't play.

I forgot to add dav1d decoder. Try the master branch code, and make sure libdav1d is installed.

What OS?

rpi os downloaded from the official site

Version of ffmpeg (or commit if its compiled from source)

system ffmpeg. hevc hw decoder is not available in upstream, and I haven't implemented my self, so system ffmpeg must be used. version is not a matter.

Which method is used for running the app? Linux build on wayland window manager or elinux?

window manager can hurt the performance a lot. elinux gbm should be the best choice. you can also stop lightdm.

[Sravdar]

@wang-bin Hello thanks for quick update. I got chance to test the master branch and now AV1 videos work, with problems.

Color channels of av1 video seems to misalligned as shown in pictures bellow.

Picture 1
Original: https://github.com/user-attachments/assets/5e6f61cc-a3aa-4534-a251-4d11ab428e42
Raspberry: https://github.com/user-attachments/assets/4c888a07-bf19-485a-9472-7a9f06b45b91

Picture 2
Original: https://github.com/user-attachments/assets/3a5f0f85-798a-4605-ab57-eeaeee06b5c2
Raspberry: https://github.com/user-attachments/assets/deb4f79b-360b-49fb-9eb5-dcb13375cfc6

Picture 3
Original: https://github.com/user-attachments/assets/effdbac1-6aff-4c0f-be47-9a2fbb99c366
Raspberry: https://github.com/user-attachments/assets/ba86e558-f2a6-497d-852f-bb943c4e06f3

For video freezing i realized that is related with the video that is being played. When i play this video whole app freezes when video playback is finished (it is expected to replay). Thats probably has something to with its poor encoding because when i encode the video again via handbrake to same resolution, fps and encoding it works without any problems. Re-encoded version of video can be found here

You can find the flutter app code here that i am testing displays.

[wang-bin]

upgrade dependencies, av1 issue should be fixed

[Sravdar]

@wang-bin Dependencies are on resolvable versions and linux packages are up-to-date. Problem still continues.

pi@pi:~/mediakit_tester $ flutter-elinux pub upgrade
Resolving dependencies... (4.2s)
Downloading packages...
  async 2.12.0 (2.13.0 available)
  characters 1.4.0 (1.4.1 available)
  fake_async 1.3.2 (1.3.3 available)
  flutter_lints 5.0.0 (6.0.0 available)
  leak_tracker 10.0.8 (11.0.1 available)
  leak_tracker_flutter_testing 3.0.9 (3.0.10 available)
  leak_tracker_testing 3.0.1 (3.0.2 available)
  lints 5.1.1 (6.0.0 available)
  material_color_utilities 0.11.1 (0.13.0 available)
  meta 1.16.0 (1.17.0 available)
  test_api 0.7.4 (0.7.6 available)
  vector_math 2.1.4 (2.2.0 available)
  vm_service 14.3.1 (15.0.2 available)
No dependencies changed.
13 packages have newer versions incompatible with dependency constraints.
Try `flutter pub outdated` for more information.

pi@pi:~/mediakit_tester $ flutter-elinux pub outdated
Showing outdated packages.
[*] indicates versions that are not the latest available.

Package Name                  Current  Upgradable  Resolvable  Latest

direct dependencies: all up-to-date.

dev_dependencies:
flutter_lints                 *5.0.0   *5.0.0      *5.0.0      6.0.0

transitive dependencies:
async                         *2.12.0  *2.12.0     *2.12.0     2.13.0
characters                    *1.4.0   *1.4.0      *1.4.0      1.4.1
material_color_utilities      *0.11.1  *0.11.1     *0.11.1     0.13.0
meta                          *1.16.0  *1.16.0     *1.16.0     1.17.0
vector_math                   *2.1.4   *2.1.4      *2.1.4      2.2.0

transitive dev_dependencies:
fake_async                    *1.3.2   *1.3.2      *1.3.2      1.3.3
leak_tracker                  *10.0.8  *10.0.8     *10.0.8     11.0.1
leak_tracker_flutter_testing  *3.0.9   *3.0.9      *3.0.9      3.0.10
leak_tracker_testing          *3.0.1   *3.0.1      *3.0.1      3.0.2
lints                         *5.1.1   *5.1.1      *5.1.1      6.0.0
test_api                      *0.7.4   *0.7.4      *0.7.4      0.7.6
vm_service                    *14.3.1  *14.3.1     *14.3.1     15.0.2
You are already using the newest resolvable versions listed in the 'Resolvable' column.
Newer versions, listed in 'Latest', may not be mutually compatible.

[wang-bin]

upgrade libmdk

[Sravdar]

@wang-bin Thank you, i've missed that part.
Building with FVP_DEPS_LATEST=1 fixed the av1 problem.

When hardware accelerated (libffmpeg.so.8 deleted from bundle) problem with app freezing on specific h264 video continues. Now while It doesn't cause any lag or freezes when not hardware accelerated it still heats up the pi to high tempatures due to software decoding.

[wang-bin]

When hardware accelerated (libffmpeg.so.8 deleted from bundle) problem with app freezing on specific h264 video continues

You have to wait for a few days until I have a chance to test on rpi4 device

[wang-bin]

i can reproduce the freeze at ffmpeg's avcodec_receive_frame, i don't know the reason yet