No depth or infra streams in jetson AGX after applying kernel patches

[jeferal]

Hello,

We want to use Realsense D455 cameras in a jetson AGX with jetpack 6.1. We have applied the kernel patches for jetpack 6.1 that following this:

https://github.com/IntelRealSense/realsense_mipi_platform_driver/blob/master/README_JP6.md#build-environment-prerequisites

After applying the kernel patches we can detect the realsense cameras using rs-enumerate-devices.
However, when we launch our ros driver application, there are no frames from the DEPTH and INFRA streams. There are frames from COLOR and ACCEL, GYRO streams. During startup, the following errors are shown in dmesg:

[ 4345.111270] usb 2-2.4: Found UVC 1.50 device Intel(R) RealSense(TM) Depth Camera 455  (8086:0b5c)
[ 4345.558258] hid-sensor-hub 0003:8086:0B5C.0025: No report with id 0xffffffff found
[ 4347.387868] usb 2-2.4: Failed to query (GET_CUR) UVC control 2 on unit 3: -32 (exp. 1).
[ 4347.458746] usb 2-2.4: Failed to query (GET_CUR) UVC control 2 on unit 3: -32 (exp. 1).
[ 4347.962258] usb 2-2.4: Failed to query (GET_CUR) UVC control 1 on unit 3: -32 (exp. 1024).
[ 4348.018421] loop0: detected capacity change from 0 to 8
[ 4348.036048] hid-sensor-hub 0003:8086:0B5C.0025: No report with id 0xffffffff found
[ 4348.320378] usb 2-2.4: Failed to query (GET_CUR) UVC control 1 on unit 3: -32 (exp. 1024).
[ 4348.473604] usb 2-2.4: Failed to query (GET_CUR) UVC control 1 on unit 3: -32 (exp. 1024).
[ 4348.524883] usb 2-2.4: Failed to query (GET_CUR) UVC control 1 on unit 3: -32 (exp. 1024).
[ 4349.084722] usb 2-2.4: Failed to query (GET_CUR) UVC control 1 on unit 3: -32 (exp. 1024).
[ 4349.136027] usb 2-2.4: Failed to query (GET_CUR) UVC control 1 on unit 3: -32 (exp. 1024).
[ 4349.187254] usb 2-2.4: Failed to query (GET_CUR) UVC control 1 on unit 3: -32 (exp. 1024).
[ 4349.340260] usb 2-2.4: Failed to query (GET_CUR) UVC control 1 on unit 3: -32 (exp. 1024).
[ 4349.391500] usb 2-2.4: Failed to query (GET_CUR) UVC control 1 on unit 3: -32 (exp. 1024).
[ 4349.544555] usb 2-2.4: Failed to query (GET_CUR) UVC control 1 on unit 3: -32 (exp. 1024).
[ 4349.595879] usb 2-2.4: Failed to query (GET_CUR) UVC control 1 on unit 3: -32 (exp. 1024).
[ 4349.748760] usb 2-2.4: Failed to query (GET_CUR) UVC control 1 on unit 3: -32 (exp. 1024).
[ 4349.799999] usb 2-2.4: Failed to query (GET_CUR) UVC control 1 on unit 3: -32 (exp. 1024).

Do you have any clue on what these errors are?

I get the same errors when I execute the application:

rs-hello-realsense

Thank you very much for your help in advance.

[MartyG-RealSense]

Hi @jeferal Have you installed the RealSense ROS wrapper and launched it, please? Depth and infrared topics would not be published unless the wrapper is first launched.

https://github.com/IntelRealSense/realsense-ros

[jeferal]

We have our own ROS application that uses librealsense. It has been tested thoroughly with multicam configuration in x86 machines.

If possible, I would focus first in why do we get errors like this:

[ 4348.320378] usb 2-2.4: Failed to query (GET_CUR) UVC control 1 on unit 3: -32 (exp. 1024).

When executing an official program like this:

rs-hello-realsense

I am afraid this could mean there is something wrong with the kernel configuration, but I would like some clues if possible.
Could you explain me possible reasons of this error, please? Thanks a lot for your quick reply.

[MartyG-RealSense]

In 2020 one of my RealSense colleagues provided the following explanation about this error.

---

The message uvcvideo: Failed to query (GET_CUR) UVC control indicates a failure on (kernel) driver level to read the requested parameter, for instance if the device was busy with internal operation or that the incoming requests queue is full. In most cases this message can be disregarded as the SDK has built-in auto-retry mechanism to compensate for the temporalities.

---

Which librealsense SDK version are you using, please? Official support for JetPack 6.1 was only very recently added in the current latest 2.57.3 beta SDK version.

Before that, the SDK was able to work on Jetsons with JetPack 6.1 if the SDK was built from source code using the libuvc backend installation method described at #13643 (comment)

A libuvc build of the SDK bypasses the kernel and so is less likely to experience errors related to conflicts between librealsense and the kernel and can work with RealSense camera models equipped with an IMU (such as D455). If you are using the IMU-equipped camera with JP6 on a USB connection (which you would be on a D455) then it would be unnecessary to install the RealSense MIPI driver as libuvc enables the D455 to be detected and accessed on a Jetson with JP6.

In some cases though, the camera may still not be detected by a libuvc build of the SDK. In that situation, taking the further step of downgrading the camera firmware version to 5.13.0.50 resolves the issue and enables the camera to work correctly in most cases. A D455 must not be downgraded to a firmware older than 5.13.0.50 though or it will cease functioning as 5.13.0.50 is the minimum firmware that supports D455.

[jeferal]

Thank you very much for all the detailed information, I really appreaciate it. We are using this version of librealsense: 2.50.0 built from source using FORCE_RSUSB_BACKEND OFF (therefore using V4L if I am not wrong).
The firmware version of the cameras we have is: 05.13.00.50

[MartyG-RealSense]

Yes, if RSUSB is set to Off then the SDK is using the V4L2 Backend (also known as the native backend). This type of SDK installation communicates with the kernel.

FORCE_RSUSB_BACKEND was the successor of LIBUVC_BACKEND, though they work similarly and so using either flag is fine. An RSUSB or libuvc backend = On build of the SDK is not dependent on a particular Linux version or kernel version and does not require a kernel patch to be applied.

Some prefer the kernel-connected V4L2 backend though for the advantages that it provides. #5212 (comment) has a good comparison of the advantages and disadvantages of a libuvc-based install vs kernel patching. Scroll down to the section of the linked-to comment headed What are the advantages and disadvantages of using libuvc vs patched kernel modules?

[jeferal]

Thank you very much for your help @MartyG-RealSense. In the end the issue was that that I was not reading the frame metadata correctly and some logic afterwards was failiing.
Apparently, I can't read metadata fields from color stream, depth and infra streams such as:

• frame_counter
• exposure
• emitter status

Has reading this metadata validated for the kernel patches in the jetson AGX for jetpack 6.1? Is there any rs tool I could use to minimally test reading this metadata? Than you again for your help.

[MartyG-RealSense]

You are very welcome!

I am not aware of issues relating to problems accessing metadata with Jetson and JP6 specifically.

You can check in the RealSense Viewer whether metadata is available and being provided by left-clicking on the 'Show Frame Metadata' icon on top of the stream panel (highlighted on the image below) to overlay metadata such as the frame count and emitter status.

If metadata support has not been enabled then the frame count will be shown but the emitter status (Frame Laser Power Mode for the emitter mode and Frame Laser Power for the emitter Laser Power value) will not.

You can display metadata for the infrared and RGB streams by clicking the Show Frame Metadata icon on top of their stream panels.

In general, on an VL42 backend build of the SDK, available metadata will be limited unless a kernel patch script (specifically, the L4T one on Jetson) because metadata support will be disabled until the patch is applied.