drivers: video: board and sensor testing

[josuah]

A thread to publish test results about various boards being tested, feel free to add your own test results: this will help maintainers and users alike.

First, select a branch such as main or a custom PR you published and get the commit hash and copy it:

git rev-parse HEAD

Testing with USB

USB Video Class (UVC) is now merged. To build the new UVC sample for your board (eventually with an extra --shield <shieldname>)

west build --board <boardname> samples/subsys/usb/uvc/

Testing with a display

west build --board <boardname> samples/drivers/video/capture/

Testing with a ethernet/wifi

west build --board <boardname> samples/drivers/video/tcpserversink/

Publishing the results

Then you can publish the results back here to help anyone who want to know what worked on which commit.

For instance:

66eb61ae38958e133a4f844f63da46dfe91fa336
```
west build --board frdm_mcxn947/mcxn947/cpu0 --shield dvp_20pin_ov7670 sample/subsys/usb/uvc
```
It works but I see some horizontal line defects through the image.

[josuah]

Tests planned

• weact_mini_stm32h743 for the weact_ov2640_cam_module shield
• mimxrt1160_evk for MIPI/CSI sensors
• arduino_giga_r1 for all the Arducam shields
• st_stm32f746g_disco with st_b_cams_omv_mb1683 and all the ST camera shields
• seeed_xiao_ble/sense for ESP32 LCDCAM driver

P.S.: The selection of boards only reflects what authors randomly had on their desk at a particular point in time, and what is easily available for order, and is not an official recommendation from Zephyr Project or the video maintainers.

[josuah]

Vendor or user alike, wanting your board tested? Wanting a board ported? Do say so in the conversation below or anywhere else!

[josuah]

arduino_nicla_vision

66eb61a

west build -b arduino_nicla_vision/stm32h747xx/m7 samples/subsys/usb/uvc/

Red and blue channel are inverted so the colors look a bit off (wood is brown, not dark blue), a known bug.

• drivers: video: gc2145: Red/Blue channel inversion #89567

[josuah]

The Nicla Vision is currently affected by this issue:

• drivers: video: dcmi: DMA error message on capture sample #92015

Here is a quickfix in the meantime:

• drivers: video: dcmi: fix DMA channel configuration #92018

But on the upside, this PR fixes the red/blue swap:

• GC2145: fixes (fix of B/R (U/V) color swap) / cleanups, cci API usage and allow driver multi-instance #91991

[josuah]

For anyone wanting to test UVC on Windows with the default "Camera" application, it is necessary to reduce the resolution:

This is due to the fact that currently, video drivers do not limit their own resolution to accommodate the video buffer size, and will happily configure frames sizes larger than their entire RAM... To decide if this is to be fixed in UVC or Zephyr Video.

On Linux, ffplay /dev/video2 (or other number) works out of the box.

MacOSX testing/fixing in progress. #92320

P.S. you need to apply this fix first, which should be merged very soon. Merged!

[josuah]

On VLC on Windows, same recommendation applies: manually setting the desired resolution might be needed:

[josuah]

frdm_mcxn947

66eb61a

west build --board frdm_mcxn947/mcxn947/cpu0 --shield dvp_20pin_ov7670 samples/subsys/usb/uvc/

Not sure if that problem is specific to UVC or not but...

With the camera covered (dark environment)

With the camera exposed to light:

As soon as the video is stopped, there is a bug:

*** Booting Zephyr OS build v4.1.0-4598-ge6a230015b0f ***
[00:00:00.044,000] <inf> uvc_sample: Waiting the host to select the video format
[00:00:07.658,000] <wrn> usbd_uvc: Could not set the framerate of video-sdma
[00:00:07.659,000] <inf> uvc_sample: The host selected format 'RGBP' 320x240, preparing 2 buffers of 153600 bytes
[00:00:07.659,000] <wrn> uvc_sample: Failed to setup the signal on video-sdma output endpoint
[00:00:07.659,000] <inf> uvc_sample: Starting the video transfer
[00:00:12.581,000] <err> os: ***** HARD FAULT *****
[00:00:12.581,000] <err> os:   Bus fault on vector table read
[00:00:12.581,000] <err> os: r0/a1:  0xfefa125b  r1/a2:  0x10013197  r2/a3:  0x30000000
[00:00:12.581,000] <err> os: r3/a4:  0x1000d511 r12/ip:  0x30003b24 r14/lr:  0x00000000
[00:00:12.581,000] <err> os:  xpsr:  0x00000008
[00:00:12.581,000] <err> os: Faulting instruction address (r15/pc): 0x00000000
[00:00:12.581,000] <err> os: >>> ZEPHYR FATAL ERROR 0: CPU exception on CPU 0
[00:00:12.581,000] <err> os: Fault during interrupt handling

[00:00:12.581,000] <err> os: Current thread: 0x300028a8 (unknown)
[00:00:12.582,000] <err> os: Halting system

addr2line -afip -e build/zephyr/zephyr.elf 0xfefa125b 0x1000d511 0x10013197 0x30003b24 0x30000000
0xfefa125b: ?? ??:0
0x1000d511: idle at /home/tinyvision/zephyrproject/zephyr/kernel/idle.c:23
0x10013197: USB_DeviceNotificationTrigger at /home/tinyvision/zephyrproject/zephyr/drivers/usb/udc/udc_mcux_ehci.c:541
0x30003b24: $d at /home/tinyvision/zephyrproject/zephyr/kernel/init.c:46
0x30000000: $d at video_mcux_smartdma.c:?

Besides, the feed is stable, so not much is missing and it is still possible to test the video driver that way (the image reacts to light).

[josuah]

After applying a needed board jumper rework, the colors look a lot more reasonable, but the empty dark lines remain visible.

Image upscaled 4x horizontally and 4x vertically:

Note: UVC was not yet fully proven to work with partial frames such as what the SmartDMA driver produces.

[ngphibang]

I think it's simpler to test on the video capture sample ? There was 3 issues that I noticed on this board:

1. Format should be set to RGB565 QVGA by default, fixed by drivers: video: ov7670: Set default format to RGB565 QVGA #90282 (merged)
2. The CPU halt problem is due to vbuf null pointer, fixed by 55eb71f (merged)
3. After all, the smartdma still cannot dequeue the buffer (no bug but no frame to dequeue), it stucks at https://github.com/zephyrproject-rtos/zephyr/blob/main/drivers/video/video_mcux_smartdma.c#L176

I tried zephyr v4.1.0 and hal nxp at fad8c6d49c2bc22c2056b291f82803e8da701806 (just before sdk next gen) but the problem 3 still persists. That means it seems not because recent changes in video system on the Zephyr side. I think the problem is due to the smartdma low level driver or something else on NXP SDK side. I cannot check further.

I created issue here: nxp-mcuxpresso/mcuxsdk-core#3

[josuah]

Thank you so much! Good point about the display, it seems how it is currently tested, good to reproduce the results from NXP.
Eager to test more sensors with it. : )

[josuah]

This got fixed on nxp-mcuxpresso/mcuxsdk-core#3!

git remote add nxp https://github.com/nxp-upstream/zephyr/
git fetch nxp

git remote add tvai https://github.com/nxp-upstream/zephyr/
git fetch tvai

git checkout pr-usb-uvc
git cherry-pick nxp/fix-ov7670-issue

west build --board frdm_mcxn947/mcxn947/cpu0 --shield dvp_20pin_ov7670 samples/subsys/usb/uvc/

[josuah]

Both of pr-usb-uvc, fix-ov7670-issue are now upstream. Some issue reported with CONFIG_LOG_MODE_IMMEDIATE=y though it seems possible to get it working by default.

Sometimes, several reboot are required to get the UVC feed through, under investigation.

[josuah]

native_sim

66eb61a

west build --boart native_sim/native/64 samples/drivers/video/capture_to_lvgl

The statistics overlay are not printed on top of the video frame anymore. Probably something to fix in main.c?

Otherwise the capture-to-lvgl sample still works.

[NinoSc]

LV_USE_PERF_MONITOR depends on LV_USE_SYSMON. Should be enough to add CONFIG_LV_USE_SYSMON and CONFIG_LV_USE_DROPDOWN to the prj.conf!

[herculanodavi]

esp32s3_eye

1fbb4e8

west build -p -b esp32s3_eye/esp32s3/procpu zephyr/samples/drivers/video/capture

Works but the image is flipped vertically, need to set the VIDEO_CID_VFLIP option.

[josuah]

Thank you very much for the testing! First Espressif board of this board.
Espressif LCDCAM driver is now confirmed still functional.

[herculanodavi]

esp32s3_eye

dda019a

west build -p -b esp32s3_eye/esp32s3/procpu zephyr/samples/drivers/video/capture_to_lvgl

FPS menu is drawn ok, but captured image is flickering.

[josuah]

Thank you for the test! Interesting to see the difference between the capture and capture_to_lvgl sample...
The FPS counter being ok suggests it is not just a pixel format error on the display side (there's an inversion with a bugfix coming).

[josuah]

frdm_mcxn236

#92058 after rebasing on top of 4.2-rc0

west build --board frdm_mcxn236/236 --shield dvp_20pin_ov7670 sample/subsys/usb/uvc

It needs to be started several times, occasionally some artifacts where groups of several lines are shuffled around, other than that, stable image coming through.