drivers: video: switch from custom buffer to RTIO

[josuah]

Introduction

Switch ad-hoc video buffers and submission/completion queues (fifo_in/fifo_out fields) + polling API to RTIO.

Problem description

The video enqueue()/dequeue()/set_signal() operations use a custom struct for video buffers and FIFO storage. Furthermore, the polling API is rarely implemented and not used in the capture example.

Proposed change

Replace the submission/completion mechanism of Video by RTIO, benefiting from reduced context switching, reduced maintenance, and provides a polling API implicitly without user requiring to implement it.

Detailed RFC

Proposed change (Detailed)

At a high-level, the modifications would be:

• struct video_buffer -> rtio_cqe/rtio_sqe

• video_enqueue() -> rtio_sqe_acquire()+rtio_sqe_prep_read()+rtio_submit()

• video_dequeue() -> rtio_cqe_consume()

• (struct sensor_data).fifo_in -> (struct sensor_data).io_q

• (struct sensor_data).fifo_out -> (struct sensor_data).io_q (merged "I/O operation queue")

• (*video_api_enqueue_t)() -> (*submit)()

• k_fifo_put(&data->fifo_out, vbuf); -> rtio_iodev_sqe_ok(iodev_sqe, result);

• video_set_signal() -> not needed anymore

• k_poll_signal_raise(data->signal, VIDEO_BUF_DONE); -> not needed anymore (part of rtio_iodev_sqe_ok(iodev_sqe, result);)

• k_poll_signal_raise(data->signal, VIDEO_BUF_ERROR); -> rtio_iodev_sqe_err(iodev_sqe, result);

• vbuf->extra_fields -> Beginning of the RTIO buffer payload

Dependencies

All features needed are already in Zephyr, and the switch to RTIO is a matter of progressively converting the drivers.

A "RTIO shim" strategy is frequenty used as a glue layer to cover drivers not converted yet.

Concerns and Unresolved Questions

If the migration happens, sould struct video_buffer and several other APIs be kept for compatibility or a more convenient/flexible API be exposed instead?

Alternatives

Make use of struct net_buf instead of RTIO.

[josuah]

Cc: @loicpoulain @ngphibang @avolmat-st

[josuah]

Background information on integration with RTIO

• drivers: video: API enhancement #72959 (comment)
• RTIO: Add Workqueue support for delayed work items #77100
• rtio: API version 0.2.0, unstable #89853
• Thread on #rtio on the chat server

Background information on video buffer management:

• drivers: video: sw_generator: add more formats and try to simplify #85968 (comment)

[teburd]

I'm very supportive of this, so anything that needs to happen to help let me know

[josuah]

I did not put this in action yet (I plan to start this week), but some early feedback from discussions at Embedded Recipes:

• Gstreamer developer told me their only regret with V4L2 APIs was the unability to queue a buffer in multple queues at once. For instance encoding data at multiple resolutions simultaneously to dynamicallly switch depending on the network conditions.

• Linux video ecosystem has all sorts of facilities allowing to configure a pipeline and then let the routing happen. If there is already some kind of "splice(2)" equivalent for RTIO this can be convenient

For both it can be possible to work around on top of RTIO though!

Thank you for your support!