-
Notifications
You must be signed in to change notification settings - Fork 7.6k
usb: device_next: new USB Video Class (UVC) implementation #76798
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -11,4 +11,5 @@ supported: | |
- gpio | ||
- spi | ||
- i2c | ||
- usbd | ||
vendor: arduino |
josuah marked this conversation as resolved.
Show resolved
Hide resolved
|
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,12 @@ | ||
# Copyright (c) 2025 tinyVision.ai Inc. | ||
# SPDX-License-Identifier: Apache-2.0 | ||
|
||
description: | | ||
USB Video Class (UVC) device instance. | ||
|
||
Each UVC instance added to the USB Device Controller (UDC) node will be visible | ||
as a new camera from the host point of view. | ||
|
||
compatible: "zephyr,uvc-device" | ||
|
||
include: base.yaml |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,47 @@ | ||
/* | ||
* Copyright (c) 2025 tinyVision.ai Inc. | ||
* | ||
* SPDX-License-Identifier: Apache-2.0 | ||
*/ | ||
|
||
/** | ||
* @file | ||
* @brief USB Video Class (UVC) public header | ||
*/ | ||
|
||
#ifndef ZEPHYR_INCLUDE_USB_CLASS_USBD_UVC_H | ||
#define ZEPHYR_INCLUDE_USB_CLASS_USBD_UVC_H | ||
|
||
#include <zephyr/device.h> | ||
|
||
/** | ||
* @brief USB Video Class (UVC) device API | ||
* @defgroup usbd_uvc USB Video Class (UVC) device API | ||
* @ingroup usb | ||
* @since 4.2 | ||
* @version 0.1.0 | ||
* @see uvc: "Universal Serial Bus Device Class Definition for Video Devices" | ||
* Document Release 1.5 (August 9, 2012) | ||
* @{ | ||
*/ | ||
|
||
/** | ||
* @brief Set the video device that a UVC instance will use. | ||
* | ||
* It will query its supported controls, formats and frame rates, and use this information to | ||
* generate USB descriptors sent to the host. | ||
* | ||
* At runtime, it will forward all USB controls from the host to this device. | ||
* | ||
* @note This function must be called before @ref usbd_enable. | ||
* | ||
* @param uvc_dev The UVC device | ||
* @param video_dev The video device that this UVC instance controls | ||
*/ | ||
void uvc_set_video_dev(const struct device *uvc_dev, const struct device *video_dev); | ||
|
||
kartben marked this conversation as resolved.
Show resolved
Hide resolved
|
||
/** | ||
* @} | ||
*/ | ||
|
||
#endif /* ZEPHYR_INCLUDE_USB_CLASS_USBD_UVC_H */ |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,8 @@ | ||
# SPDX-License-Identifier: Apache-2.0 | ||
|
||
cmake_minimum_required(VERSION 3.20.0) | ||
find_package(Zephyr REQUIRED HINTS $ENV{ZEPHYR_BASE}) | ||
project(usb_video) | ||
|
||
include(${ZEPHYR_BASE}/samples/subsys/usb/common/common.cmake) | ||
target_sources(app PRIVATE src/main.c) |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,9 @@ | ||
# Copyright The Zephyr Project Contributors | ||
# SPDX-License-Identifier: Apache-2.0 | ||
|
||
# Source common USB sample options used to initialize new experimental USB | ||
# device stack. The scope of these options is limited to USB samples in project | ||
# tree, you cannot use them in your own application. | ||
source "samples/subsys/usb/common/Kconfig.sample_usbd" | ||
|
||
source "Kconfig.zephyr" |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,183 @@ | ||
.. zephyr:code-sample:: uvc | ||
:name: USB Video webcam | ||
:relevant-api: usbd_api usbd_uvc video_interface | ||
|
||
Send video frames over USB. | ||
|
||
Overview | ||
******** | ||
|
||
This sample demonstrates how to use a USB Video Class instance to send video data over USB. | ||
|
||
Upon connection, a video device will show-up on the host, usable like a regular webcam device. | ||
|
||
Any software on the host can then access the video stream as a local video source. | ||
|
||
Requirements | ||
************ | ||
|
||
This sample uses the new USB device stack and requires the USB device | ||
controller ported to the :ref:`udc_api`. | ||
|
||
Building and Running | ||
******************** | ||
|
||
If a board does not have a camera supported, the :ref:`snippet-video-sw-generator` snippet can be | ||
used to test without extra hardware than the USB interface, via a software-generated test pattern: | ||
|
||
.. zephyr-app-commands:: | ||
:zephyr-app: samples/subsys/usb/uvc | ||
:board: frdm_mcxn947/mcxn947/cpu0 | ||
:snippets: video-sw-generator | ||
:goals: build flash | ||
:compact: | ||
|
||
If a board is equipped with a supported image sensor configured as the ``zephyr,camera`` chosen | ||
node, then it will be used as the video source. The sample can then be built as follows: | ||
|
||
.. zephyr-app-commands:: | ||
:zephyr-app: samples/subsys/usb/uvc | ||
:board: arduino_nicla_vision/stm32h747xx/m7 | ||
:goals: build flash | ||
:compact: | ||
|
||
The device is expected to be detected as a webcam device: | ||
|
||
.. tabs:: | ||
|
||
.. group-tab:: Ubuntu | ||
|
||
The ``dmesg`` logs are expected to mention a ``generic UVC device``. | ||
|
||
The ``lsusb`` is expected to show an entry for a Zephyr device. | ||
|
||
Refers to `Ideas on board FAQ <https://www.ideasonboard.org/uvc/faq/>`_ | ||
for how to get more debug information. | ||
|
||
.. group-tab:: MacOS | ||
|
||
The ``dmesg`` logs are expected to mention a video device. | ||
|
||
The ``ioreg -p IOUSB`` command list the USB devices including cameras. | ||
|
||
The ``system_profiler SPCameraDataType`` command list video input devices. | ||
|
||
.. group-tab:: Windows | ||
|
||
The Device Manager or USBView utilities permit to list the USB devices. | ||
|
||
The 3rd-party USB Tree View allows to review and debug the descriptors. | ||
|
||
In addition, the `USB3CV <https://www.usb.org/document-library/usb3cv>`_ tool | ||
from USB-IF can check that the device is compliant with the UVC standard. | ||
|
||
Playing the Stream | ||
================== | ||
|
||
The device is recognized by the system as a native webcam and can be used by any video application. | ||
|
||
For instance with VLC: | ||
:menuselection:`Media --> Open Capture Device --> Capture Device --> Video device name`. | ||
|
||
Or with Gstreamer and FFmpeg: | ||
|
||
.. tabs:: | ||
|
||
.. group-tab:: Ubuntu | ||
|
||
Assuming ``/dev/video0`` is your Zephyr device. | ||
|
||
.. code-block:: console | ||
ffplay -i /dev/video0 | ||
.. code-block:: console | ||
gst-launch-1.0 v4l2src device=/dev/video0 ! videoconvert ! autovideosink | ||
.. group-tab:: MacOS | ||
|
||
Assuming ``0:0`` is your Zephyr device. | ||
|
||
.. code-block:: console | ||
ffplay -f avfoundation -i 0:0 | ||
.. code-block:: console | ||
gst-launch-1.0 avfvideosrc device-index=0 ! autovideosink | ||
.. group-tab:: Windows | ||
|
||
Assuming ``UVC sample`` is your Zephyr device. | ||
|
||
.. code-block:: console | ||
ffplay.exe -f dshow -i video="UVC sample" | ||
.. code-block:: console | ||
gst-launch-1.0.exe ksvideosrc device-name="UVC sample" ! videoconvert ! autovideosink | ||
The video device can also be used by web and video call applications systems. | ||
|
||
Android and iPad (but not yet iOS) are also expected to work via dedicated applications. | ||
|
||
Accessing the Video Controls | ||
============================ | ||
|
||
On the host system, the controls would be available as video source | ||
control through various applications, like any webcam. | ||
|
||
.. tabs:: | ||
|
||
.. group-tab:: Ubuntu | ||
|
||
Assuming ``/dev/video0`` is your Zephyr device. | ||
|
||
.. code-block:: console | ||
$ v4l2-ctl --device /dev/video0 --list-ctrls | ||
Camera Controls | ||
auto_exposure 0x009a0901 (menu) : min=0 max=3 default=1 value=1 (Manual Mode) | ||
exposure_dynamic_framerate 0x009a0903 (bool) : default=0 value=0 | ||
exposure_time_absolute 0x009a0902 (int) : min=10 max=2047 step=1 default=384 value=384 flags=inactive | ||
$ v4l2-ctl --device /dev/video0 --set-ctrl auto_exposure=1 | ||
$ v4l2-ctl --device /dev/video0 --set-ctrl exposure_time_absolute=1500 | ||
.. group-tab:: MacOS | ||
|
||
The `VLC <https://www.videolan.org/vlc/>`_ client and the system Webcam Settings panel | ||
allows adjustment of the supported video controls. | ||
|
||
.. group-tab:: Windows | ||
|
||
The `VLC <https://www.videolan.org/vlc/>`_ client and `Pot Player <https://potplayer.tv/>`_ | ||
client permit to further access the video controls. | ||
|
||
Software Processing | ||
=================== | ||
|
||
Software processing tools can also use the video interface directly. | ||
|
||
Here is an example with OpenCV (``pip install opencv-python``): | ||
|
||
.. code-block:: python | ||
import cv2 | ||
# Number of the /dev/video# interface | ||
devnum = 2 | ||
cv2.namedWindow("preview") | ||
vc = cv2.VideoCapture(devnum) | ||
while (val := vc.read())[0]: | ||
cv2.waitKey(20) | ||
cv2.imshow("preview", val[1]) | ||
cv2.destroyWindow("preview") | ||
vc.release() |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,11 @@ | ||
/* | ||
* Copyright (c) 2025 tinyVision.ai Inc. | ||
* | ||
* SPDX-License-Identifier: Apache-2.0 | ||
*/ | ||
|
||
/ { | ||
uvc: uvc { | ||
compatible = "zephyr,uvc-device"; | ||
}; | ||
Comment on lines
+8
to
+10
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Just
Would it work with There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This builds and works [1]:
I should have a Nicla Vision in the mail tomorrow or so to test, but both of these build just fine:
[1]: I always have a pico at hand from another project, no preference! |
||
}; |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,2 @@ | ||
# Enough two 320x240 YUYV frames | ||
CONFIG_VIDEO_BUFFER_POOL_SZ_MAX=163840 |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,2 @@ | ||
CONFIG_VIDEO_BUFFER_POOL_SZ_MAX=40000 | ||
CONFIG_VIDEO_BUFFER_POOL_NUM_MAX=2 |
Original file line number | Diff line number | Diff line change | ||||||
---|---|---|---|---|---|---|---|---|
@@ -0,0 +1,14 @@ | ||||||||
CONFIG_LOG=y | ||||||||
CONFIG_POLL=y | ||||||||
CONFIG_SAMPLE_USBD_PID=0x0011 | ||||||||
CONFIG_SAMPLE_USBD_PRODUCT="UVC sample" | ||||||||
jfischer-no marked this conversation as resolved.
Show resolved
Hide resolved
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
Or at least something more than 1024, as 1024 seems not enough for me, causing these errors with RP2350 and Linux 6.15 host:
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Given the timestamps it seems like it happened during enumeration. There is a lot of activity for UVC during enumeration as it queries the drivers for what they support, and plenty of descriptors to transmit for complex video sources... Increasing the pool size makes sense at least to me. Applications can then optimize their resource to fit more tightly but at least the samples work on more boards by default. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yeah, this causes a enumeration error I can paste if needed whenever I'm back to the workstation There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
|
||||||||
CONFIG_UDC_BUF_POOL_SIZE=2048 | ||||||||
CONFIG_UDC_DRIVER_LOG_LEVEL_WRN=y | ||||||||
CONFIG_USBD_LOG_LEVEL_WRN=y | ||||||||
josuah marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||||
CONFIG_USBD_VIDEO_CLASS=y | ||||||||
CONFIG_USBD_VIDEO_LOG_LEVEL_WRN=y | ||||||||
josuah marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||||
CONFIG_USB_DEVICE_STACK_NEXT=y | ||||||||
CONFIG_VIDEO=y | ||||||||
CONFIG_VIDEO_BUFFER_POOL_NUM_MAX=2 | ||||||||
CONFIG_VIDEO_BUFFER_POOL_SZ_MAX=24576 | ||||||||
avolmat-st marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||||
CONFIG_VIDEO_LOG_LEVEL_WRN=y |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,29 @@ | ||
sample: | ||
name: USB Video sample | ||
common: | ||
harness: console | ||
harness_config: | ||
type: one_line | ||
regex: | ||
- "Waiting the host to select the video format" | ||
tests: | ||
sample.subsys.usb.uvc: | ||
depends_on: | ||
- usbd | ||
tags: usb video | ||
extra_args: SNIPPET=video-sw-generator | ||
integration_platforms: | ||
- nrf52840dk/nrf52840 | ||
- nrf54h20dk/nrf54h20/cpuapp | ||
- frdm_k64f | ||
- stm32f723e_disco | ||
- nucleo_f413zh | ||
- mimxrt685_evk/mimxrt685s/cm33 | ||
- mimxrt1060_evk/mimxrt1062/qspi | ||
sample.subsys.usb.uvc.camera: | ||
depends_on: | ||
- usbd | ||
tags: usb video | ||
filter: dt_chosen_enabled("zephyr,camera") | ||
integration_platforms: | ||
- arduino_nicla_vision/stm32h747xx/m7 | ||
jfischer-no marked this conversation as resolved.
Show resolved
Hide resolved
|
Uh oh!
There was an error while loading. Please reload this page.