Skip to content

usb: device_next: new USB Video Class (UVC) implementation #76798

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Jun 27, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -11,4 +11,5 @@ supported:
- gpio
- spi
- i2c
- usbd
vendor: arduino
2 changes: 2 additions & 0 deletions doc/connectivity/usb/device/usb_device.rst
Original file line number Diff line number Diff line change
Expand Up @@ -551,6 +551,8 @@ The following Product IDs are currently used:
+----------------------------------------------------+--------+
| :zephyr:code-sample:`uac2-implicit-feedback` | 0x000F |
+----------------------------------------------------+--------+
| :zephyr:code-sample:`uvc` | 0x0011 |
+----------------------------------------------------+--------+
| :zephyr:code-sample:`usb-dfu` (DFU Mode) | 0xFFFF |
+----------------------------------------------------+--------+

Expand Down
4 changes: 4 additions & 0 deletions doc/connectivity/usb/device_next/usb_device.rst
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,8 @@ Samples

* :zephyr:code-sample:`uac2-implicit-feedback`

* :zephyr:code-sample:`uvc`

Samples ported to new USB device support
----------------------------------------

Expand Down Expand Up @@ -223,6 +225,8 @@ instance (``n``) and is used as an argument to the :c:func:`usbd_register_class`
+-----------------------------------+-------------------------+-------------------------+
| Bluetooth HCI USB transport layer | :ref:`bt_hci_raw` | :samp:`bt_hci_{n}` |
+-----------------------------------+-------------------------+-------------------------+
| USB Video Class (UVC) | Video device | :samp:`uvc_{n}` |
+-----------------------------------+-------------------------+-------------------------+

CDC ACM UART
============
Expand Down
12 changes: 12 additions & 0 deletions dts/bindings/usb/zephyr,uvc-device.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
# Copyright (c) 2025 tinyVision.ai Inc.
# SPDX-License-Identifier: Apache-2.0

description: |
USB Video Class (UVC) device instance.

Each UVC instance added to the USB Device Controller (UDC) node will be visible
as a new camera from the host point of view.

compatible: "zephyr,uvc-device"

include: base.yaml
47 changes: 47 additions & 0 deletions include/zephyr/usb/class/usbd_uvc.h
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
/*
* Copyright (c) 2025 tinyVision.ai Inc.
*
* SPDX-License-Identifier: Apache-2.0
*/

/**
* @file
* @brief USB Video Class (UVC) public header
*/

#ifndef ZEPHYR_INCLUDE_USB_CLASS_USBD_UVC_H
#define ZEPHYR_INCLUDE_USB_CLASS_USBD_UVC_H

#include <zephyr/device.h>

/**
* @brief USB Video Class (UVC) device API
* @defgroup usbd_uvc USB Video Class (UVC) device API
* @ingroup usb
* @since 4.2
* @version 0.1.0
* @see uvc: "Universal Serial Bus Device Class Definition for Video Devices"
* Document Release 1.5 (August 9, 2012)
* @{
*/

/**
* @brief Set the video device that a UVC instance will use.
*
* It will query its supported controls, formats and frame rates, and use this information to
* generate USB descriptors sent to the host.
*
* At runtime, it will forward all USB controls from the host to this device.
*
* @note This function must be called before @ref usbd_enable.
*
* @param uvc_dev The UVC device
* @param video_dev The video device that this UVC instance controls
*/
void uvc_set_video_dev(const struct device *uvc_dev, const struct device *video_dev);

/**
* @}
*/

#endif /* ZEPHYR_INCLUDE_USB_CLASS_USBD_UVC_H */
3 changes: 2 additions & 1 deletion samples/subsys/usb/common/sample_usbd_init.c
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,8 @@ static void sample_fix_code_triple(struct usbd_context *uds_ctx,
IS_ENABLED(CONFIG_USBD_CDC_ECM_CLASS) ||
IS_ENABLED(CONFIG_USBD_CDC_NCM_CLASS) ||
IS_ENABLED(CONFIG_USBD_MIDI2_CLASS) ||
IS_ENABLED(CONFIG_USBD_AUDIO2_CLASS)) {
IS_ENABLED(CONFIG_USBD_AUDIO2_CLASS) ||
IS_ENABLED(CONFIG_USBD_VIDEO_CLASS)) {
/*
* Class with multiple interfaces have an Interface
* Association Descriptor available, use an appropriate triple
Expand Down
8 changes: 8 additions & 0 deletions samples/subsys/usb/uvc/CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
# SPDX-License-Identifier: Apache-2.0

cmake_minimum_required(VERSION 3.20.0)
find_package(Zephyr REQUIRED HINTS $ENV{ZEPHYR_BASE})
project(usb_video)

include(${ZEPHYR_BASE}/samples/subsys/usb/common/common.cmake)
target_sources(app PRIVATE src/main.c)
9 changes: 9 additions & 0 deletions samples/subsys/usb/uvc/Kconfig
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
# Copyright The Zephyr Project Contributors
# SPDX-License-Identifier: Apache-2.0

# Source common USB sample options used to initialize new experimental USB
# device stack. The scope of these options is limited to USB samples in project
# tree, you cannot use them in your own application.
source "samples/subsys/usb/common/Kconfig.sample_usbd"

source "Kconfig.zephyr"
183 changes: 183 additions & 0 deletions samples/subsys/usb/uvc/README.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,183 @@
.. zephyr:code-sample:: uvc
:name: USB Video webcam
:relevant-api: usbd_api usbd_uvc video_interface

Send video frames over USB.

Overview
********

This sample demonstrates how to use a USB Video Class instance to send video data over USB.

Upon connection, a video device will show-up on the host, usable like a regular webcam device.

Any software on the host can then access the video stream as a local video source.

Requirements
************

This sample uses the new USB device stack and requires the USB device
controller ported to the :ref:`udc_api`.

Building and Running
********************

If a board does not have a camera supported, the :ref:`snippet-video-sw-generator` snippet can be
used to test without extra hardware than the USB interface, via a software-generated test pattern:

.. zephyr-app-commands::
:zephyr-app: samples/subsys/usb/uvc
:board: frdm_mcxn947/mcxn947/cpu0
:snippets: video-sw-generator
:goals: build flash
:compact:

If a board is equipped with a supported image sensor configured as the ``zephyr,camera`` chosen
node, then it will be used as the video source. The sample can then be built as follows:

.. zephyr-app-commands::
:zephyr-app: samples/subsys/usb/uvc
:board: arduino_nicla_vision/stm32h747xx/m7
:goals: build flash
:compact:

The device is expected to be detected as a webcam device:

.. tabs::

.. group-tab:: Ubuntu

The ``dmesg`` logs are expected to mention a ``generic UVC device``.

The ``lsusb`` is expected to show an entry for a Zephyr device.

Refers to `Ideas on board FAQ <https://www.ideasonboard.org/uvc/faq/>`_
for how to get more debug information.

.. group-tab:: MacOS

The ``dmesg`` logs are expected to mention a video device.

The ``ioreg -p IOUSB`` command list the USB devices including cameras.

The ``system_profiler SPCameraDataType`` command list video input devices.

.. group-tab:: Windows

The Device Manager or USBView utilities permit to list the USB devices.

The 3rd-party USB Tree View allows to review and debug the descriptors.

In addition, the `USB3CV <https://www.usb.org/document-library/usb3cv>`_ tool
from USB-IF can check that the device is compliant with the UVC standard.

Playing the Stream
==================

The device is recognized by the system as a native webcam and can be used by any video application.

For instance with VLC:
:menuselection:`Media --> Open Capture Device --> Capture Device --> Video device name`.

Or with Gstreamer and FFmpeg:

.. tabs::

.. group-tab:: Ubuntu

Assuming ``/dev/video0`` is your Zephyr device.

.. code-block:: console
ffplay -i /dev/video0
.. code-block:: console
gst-launch-1.0 v4l2src device=/dev/video0 ! videoconvert ! autovideosink
.. group-tab:: MacOS

Assuming ``0:0`` is your Zephyr device.

.. code-block:: console
ffplay -f avfoundation -i 0:0
.. code-block:: console
gst-launch-1.0 avfvideosrc device-index=0 ! autovideosink
.. group-tab:: Windows

Assuming ``UVC sample`` is your Zephyr device.

.. code-block:: console
ffplay.exe -f dshow -i video="UVC sample"
.. code-block:: console
gst-launch-1.0.exe ksvideosrc device-name="UVC sample" ! videoconvert ! autovideosink
The video device can also be used by web and video call applications systems.

Android and iPad (but not yet iOS) are also expected to work via dedicated applications.

Accessing the Video Controls
============================

On the host system, the controls would be available as video source
control through various applications, like any webcam.

.. tabs::

.. group-tab:: Ubuntu

Assuming ``/dev/video0`` is your Zephyr device.

.. code-block:: console
$ v4l2-ctl --device /dev/video0 --list-ctrls
Camera Controls
auto_exposure 0x009a0901 (menu) : min=0 max=3 default=1 value=1 (Manual Mode)
exposure_dynamic_framerate 0x009a0903 (bool) : default=0 value=0
exposure_time_absolute 0x009a0902 (int) : min=10 max=2047 step=1 default=384 value=384 flags=inactive
$ v4l2-ctl --device /dev/video0 --set-ctrl auto_exposure=1
$ v4l2-ctl --device /dev/video0 --set-ctrl exposure_time_absolute=1500
.. group-tab:: MacOS

The `VLC <https://www.videolan.org/vlc/>`_ client and the system Webcam Settings panel
allows adjustment of the supported video controls.

.. group-tab:: Windows

The `VLC <https://www.videolan.org/vlc/>`_ client and `Pot Player <https://potplayer.tv/>`_
client permit to further access the video controls.

Software Processing
===================

Software processing tools can also use the video interface directly.

Here is an example with OpenCV (``pip install opencv-python``):

.. code-block:: python
import cv2
# Number of the /dev/video# interface
devnum = 2
cv2.namedWindow("preview")
vc = cv2.VideoCapture(devnum)
while (val := vc.read())[0]:
cv2.waitKey(20)
cv2.imshow("preview", val[1])
cv2.destroyWindow("preview")
vc.release()
11 changes: 11 additions & 0 deletions samples/subsys/usb/uvc/app.overlay
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
/*
* Copyright (c) 2025 tinyVision.ai Inc.
*
* SPDX-License-Identifier: Apache-2.0
*/

/ {
uvc: uvc {
compatible = "zephyr,uvc-device";
};
Comment on lines +8 to +10
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just

uvc: uvc {
	compatible = "zephyr,uvc-device";
};

Would it work with arduino_nicla_vision_stm32h747xx_m7 out of the box?

Copy link
Collaborator Author

@josuah josuah Feb 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This builds and works [1]:

west build --board rpi_pico samples/subsys/usb/uvc/ -- \
    -DEXTRA_DTC_OVERLAY_FILE=$PWD/samples/subsys/usb/uvc/video-emul.overlay

I should have a Nicla Vision in the mail tomorrow or so to test, but both of these build just fine:

west build -p --board arduino_nicla_vision/stm32h747xx/m7 samples/subsys/usb/uvc/ -- \
    -DEXTRA_DTC_OVERLAY_FILE=$PWD/samples/subsys/usb/uvc/video-emul.overlay
west build -p --board arduino_nicla_vision/stm32h747xx/m7 samples/subsys/usb/uvc/ -- \
    -DEXTRA_DTC_OVERLAY_FILE=$PWD/samples/subsys/usb/uvc/video-dcmi.overlay

[1]: I always have a pico at hand from another project, no preference!

};
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
# Enough two 320x240 YUYV frames
CONFIG_VIDEO_BUFFER_POOL_SZ_MAX=163840
2 changes: 2 additions & 0 deletions samples/subsys/usb/uvc/boards/frdm_mcxn947_mcxn947_cpu0.conf
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
CONFIG_VIDEO_BUFFER_POOL_SZ_MAX=40000
CONFIG_VIDEO_BUFFER_POOL_NUM_MAX=2
14 changes: 14 additions & 0 deletions samples/subsys/usb/uvc/prj.conf
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
CONFIG_LOG=y
CONFIG_POLL=y
CONFIG_SAMPLE_USBD_PID=0x0011
CONFIG_SAMPLE_USBD_PRODUCT="UVC sample"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
CONFIG_SAMPLE_USBD_PRODUCT="UVC sample"
CONFIG_SAMPLE_USBD_PRODUCT="UVC sample"
CONFIG_UDC_BUF_POOL_SIZE=8192

Or at least something more than 1024, as 1024 seems not enough for me, causing these errors with RP2350 and Linux 6.15 host:

[00:00:00.338,000] <err> udc: Failed to allocate net_buf 952
[00:00:00.338,000] <err> usbd_ch9: Buffer for data|status is missing
[00:00:00.338,000] <err> usbd_ch9: Malformed setup packet
[00:00:00.338,000] <err> udc: Failed to allocate net_buf 952
[00:00:00.338,000] <err> usbd_ch9: Buffer for data|status is missing
[00:00:00.338,000] <err> usbd_ch9: Malformed setup packet
[00:00:00.338,000] <err> udc: Failed to allocate net_buf 952
[00:00:00.338,000] <err> usbd_ch9: Buffer for data|status is missing
[00:00:00.338,000] <err> usbd_ch9: Malformed setup packet
[00:00:00.342,000] <wrn> udc: Spurious suspend/resume event

Copy link
Collaborator Author

@josuah josuah Jun 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given the timestamps it seems like it happened during enumeration.

There is a lot of activity for UVC during enumeration as it queries the drivers for what they support, and plenty of descriptors to transmit for complex video sources... Increasing the pool size makes sense at least to me. Applications can then optimize their resource to fit more tightly but at least the samples work on more boards by default.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, this causes a enumeration error I can paste if needed whenever I'm back to the workstation

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CONFIG_UDC_BUF_POOL_SIZE=2048 works for me

CONFIG_UDC_BUF_POOL_SIZE=2048
CONFIG_UDC_DRIVER_LOG_LEVEL_WRN=y
CONFIG_USBD_LOG_LEVEL_WRN=y
CONFIG_USBD_VIDEO_CLASS=y
CONFIG_USBD_VIDEO_LOG_LEVEL_WRN=y
CONFIG_USB_DEVICE_STACK_NEXT=y
CONFIG_VIDEO=y
CONFIG_VIDEO_BUFFER_POOL_NUM_MAX=2
CONFIG_VIDEO_BUFFER_POOL_SZ_MAX=24576
CONFIG_VIDEO_LOG_LEVEL_WRN=y
29 changes: 29 additions & 0 deletions samples/subsys/usb/uvc/sample.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
sample:
name: USB Video sample
common:
harness: console
harness_config:
type: one_line
regex:
- "Waiting the host to select the video format"
tests:
sample.subsys.usb.uvc:
depends_on:
- usbd
tags: usb video
extra_args: SNIPPET=video-sw-generator
integration_platforms:
- nrf52840dk/nrf52840
- nrf54h20dk/nrf54h20/cpuapp
- frdm_k64f
- stm32f723e_disco
- nucleo_f413zh
- mimxrt685_evk/mimxrt685s/cm33
- mimxrt1060_evk/mimxrt1062/qspi
sample.subsys.usb.uvc.camera:
depends_on:
- usbd
tags: usb video
filter: dt_chosen_enabled("zephyr,camera")
integration_platforms:
- arduino_nicla_vision/stm32h747xx/m7
Loading