Skip to content

Video RTIO demo #92566

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

josuah
Copy link
Contributor

@josuah josuah commented Jul 2, 2025

In some area, RTIO is used to increase performance at a slight extra cost in code size.

For video, it seems like a partial RTIO-like flow was already adopted by @loicpoulain in video APIs out of kernel primitives (the same week RTIO was introduced), so switching to RTIO actually reduces the amount of code in drivers and simplify the video APIs.

 drivers/video/Kconfig                    |   1 +
 drivers/video/video_common.c             |  25 +++++-
 drivers/video/video_sw_generator.c       |  99 ++++++-----------------
 include/zephyr/drivers/video.h           | 192 +++++++++-----------------------------------
 samples/drivers/video/capture/src/main.c |  48 +++++++----
 5 files changed, 119 insertions(+), 246 deletions(-)

API functions removed as now available through RTIO APIs:

  • .enqueue
  • .dequeue
  • .flush
  • .set_signal (the polling API)

API function added:

  • .iodev_submit

The sample main.c did not go down in size. This initial PR adds a naive implementation that exposes all the RTIO internals directly, allowing discussion with the RTIO developers and Video developers and users about what wrappers to implement.

Keeping the low-level exposed is also a good opportunity to allow application libraries to wrap video APIs in higher-level and efficient userspace APIs.

Signed-off-by: Josuah Demangeon <me@josuah.net>
@josuah josuah added area: Video Video subsystem area: RTIO labels Jul 2, 2025
@josuah
Copy link
Contributor Author

josuah commented Jul 2, 2025

Current status:

  • native_sim/native/64 works.
  • CI does not work as zephyr,video-emul-rx is not converted yet.

scrot_20250703_003852_1230x342

@josuah
Copy link
Contributor Author

josuah commented Jul 2, 2025

For anyone investigating RTIO, the implementation of the engine is actually nimble!

Other useful resources:

And the ongoing discussionso with RTIO developers (for grasping the RTIO model), and Zephyr/Linux develoeprs (for grasping the V4L2 model) through this year!

Copy link

sonarqubecloud bot commented Jul 2, 2025

@ngphibang
Copy link
Contributor

Thank you for the this implementation. I tried to quickly implement the necessary stuffs to generalize this into the framework in #92370. It is faster to illustrate the idea, hope you don't mind.

@josuah
Copy link
Contributor Author

josuah commented Jul 4, 2025

I tried to quickly implement the necessary stuffs to generalize this into the framework in #92370. It is faster to illustrate the idea, hope you don't mind.

I left the user-side APIs very crude on purpose to make it easier to design an API on #92370

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants