Skip to content

Video: addition of the imx335 driver #88825

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
May 22, 2025
Merged

Video: addition of the imx335 driver #88825

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
7a57a31
dts: bindings: video: imx335: addition of imx335 description
Apr 2, 2025
cf9aa8e
drivers: video: introduction of imx335 sensor driver
Apr 2, 2025
d64f4dd
dts: bindings: addition of RaspberryPi CSI 22 pins camera connector
May 9, 2025
83e9d4b
shields: st_b_cams_imx_mb1854: add imx335 based MB1854 board shield
Apr 2, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions boards/shields/st_b_cams_imx_mb1854/Kconfig.shield
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# Copyright (c) 2025 STMicroelectronics.
# SPDX-License-Identifier: Apache-2.0

config SHIELD_ST_B_CAMS_IMX_MB1854
def_bool $(shields_list_contains,st_b_cams_imx_mb1854)
51 changes: 51 additions & 0 deletions boards/shields/st_b_cams_imx_mb1854/doc/index.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
.. _st_b_cams_imx_mb1854:

ST B-CAMS-IMX-MB1854
####################

Overview
********

The B-CAMS-IMX camera module provides a compelling hardware set to
handle multiple computer vision scenarios and use cases. It features
a high-resolution 5‑Mpx RGB CMOS image sensor, an inertial motion unit,
and a Time‑of‑Flight sensor. It can be used with any STM32 boards featuring
a MIPI CSI-2® interface with a 22‑pin FFC connector to enable full-featured
computer vision on STM32 microcontrollers and microprocessors easily.

.. figure:: st_b_cams_imx.webp
:width: 600px
:align: center
:alt: B-CAMS-IMX-MB1854

B-CAMS-IMX MB1854 Image (Credit: STMicroelectronics.)

Requirements
************

The camera module bundle is compatible with all STM32 Discovery kits and
Evaluation boards featuring a 22 pins FFC connector, such as the STM32N6570_DK
Discovery kit.

Usage
*****

The shield can be used in any application by setting ``SHIELD`` to
``st_b_cams_imx_mb1854`` for boards with the necessary device tree node labels.

Set ``--shield "st_b_cams_imx_mb1854"`` when you invoke ``west build``. For example:

.. zephyr-app-commands::
:zephyr-app: samples/drivers/video/capture
:board: stm32n6570_dk
:shield: st_b_cams_imx_mb1854
:goals: build

References
**********

- `Product page <https://www.st.com/en/evaluation-tools/b-cams-imx.html>`_

- `Databrief <https://www.st.com/resource/en/data_brief/b-cams-imx.pdf>`_

- `User manual <https://www.st.com/resource/en/user_manual/um3354-camera-module-bundle-for-stm32-boards-stmicroelectronics.pdf>`_
Binary file added boards/shields/st_b_cams_imx_mb1854/doc/st_b_cams_imx.webp
View file
Open in desktop
Binary file not shown.
57 changes: 57 additions & 0 deletions boards/shields/st_b_cams_imx_mb1854/st_b_cams_imx_mb1854.overlay
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
/*
* Copyright (c) 2025 STMicroelectronics.
*
* SPDX-License-Identifier: Apache-2.0
*
*/

#include <zephyr/dt-bindings/gpio/raspberrypi-csi-22pins-connector.h>
#include <zephyr/dt-bindings/video/video-interfaces.h>

/ {
chosen {
zephyr,camera = &csi_22pins_capture_port;
};

imx335_input_clock: imx335-input-clock {
compatible = "fixed-clock";
clock-frequency = <24000000>;
#clock-cells = <0>;
};
};

&csi_22pins_interface {
status = "okay";
};

&csi_22pins_ep_in {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[non-blocking: discussion]

I went with such approach at first, but got some recommendation from @ngphibang:

I think the right way is keeping the video sdma node and label it as a dvp_20pin_interface as in https://github.com/zephyrproject-rtos/zephyr/blob/main/boards/nxp/mimxrt1064_evk/mimxrt1064_evk.dts#L352-#L354

The endpoint should be inside the interface node.

Copy link
Collaborator Author

@avolmat-st avolmat-st May 21, 2025 •
edited by josuah
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

First commenting on this one, seems to be the one that could lead to discussions. In our case, this is being used on the STM32N6 for example and the DCMIPP has 3 output endpoints (one for each pipe). So the DT node looks like something like that:

csi_22pins_interface: dcmipp {
    ports {
        port@0 {
             csi_22pins_ep_in: endpoint {
                 here the endpoint which is connected to the sensor endpoint
             }
        }
        port@1 {
             endpoint@0 {
                 first output endpoint / pipe of the dcmipp
             }
             csi_22pins_capture_port: endpoint@1 {
                 2nd output endpoint / pipe of the dcmipp
             }
             endpoint@2 {
                 3rd output endpoint / pipe of the dcmipp
             }
         }
  }

So here, there are actually plenty of information I need in order to "connect" the shield to the dcmipp.
1st, I need to set to enable the dcmipp node (it can't be always enabled, it should only be when there is a sensor connected to it). Hence the "interface" label: csi_22pins_interface.
Then I need to configure the input endpoint of the dcmipp and link it to the sensor endpoint, hence csi_22pins_ep_in.
I need to add the sensor to the right i2c interface, hence csi_22pins_i2c
Then I need to select an output pipe, here the 2nd one is selected via csi_22pins_capture_port and zephyr,camera = csi_22pins_capture_port.

I have the feeling that it is actually quite complicated, due to the fact that there are multiple output endpoints which are part of a 2nd port.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ouch, formatting is gone ... sorry about that.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(I added the formatting back)
Yes somehow the csi_22pins_ep_in allowed the shield to know which endpoint to select.

Copy link
Collaborator

@ngphibang ngphibang May 21, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In this case, I think you could put all the port@1 (with 3 endpoints) into the SoC devicetree, not in the shield because they are "fixed" and just the port@0 (which is flexible, which connects to the sensor) into the shield, by reserving the port@0 firts in the SoC somehow like :

				port@0 {
					reg = <1>;
				};

https://github.com/zephyrproject-rtos/zephyr/blob/main/dts/arm/nxp/nxp_rt11xx.dtsi#L920-#L943

no ?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ngphibang In this case, I think you could put all the port@1 (with 3 endpoints) into the SoC devicetree, not in the shield because they are "fixed" and just the port@0 (which is flexible, which connects to the sensor) into the shield, by reserving the port@0 firts in the SoC somehow like :

This would be a convention that drivers take to make it easy for shields...
We need some place where to document that to help new drivers being written.
Some kind of "video driver author handbook" in the long run too.
I will ask where to insert it in the docs.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

An "Implementing..." section in the (hopefully already existing) driver class page? https://docs.zephyrproject.org/latest/hardware/peripherals/sensor/index.html#implementing-sensor-drivers -- @kartben

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Honestly, I still don't really understand why we need to put &csi_22pins_ep_in outside the interface node :-) ... but it's just for my knowdledge

Copy link
Collaborator Author

@avolmat-st avolmat-st May 22, 2025 •
edited by josuah
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we can guarantee that on all board having a RPi 22 pins CSI connector the interface IP "input" endpoint (to which to connect the sensor endpoint) is going to be at:

csi_22pins_interface {
            ports {
                   port@0 {
                         endpoint {
                                // HERE
                         };
                   }
            }
}

then there is no need for such label indeed, but can we guarantee that ?
If yes then then only csi_22pins_interface is enough.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see now. Thank you !

remote-endpoint-label = "imx335_ep_out";
bus-type = <VIDEO_BUS_TYPE_CSI2_DPHY>;
data-lanes = <1 2>;
};

&csi_22pins_i2c {
imx335: camera@1a {
compatible = "sony,imx335";
clocks = <&imx335_input_clock>;
reg = <0x1a>;
reset-gpios = <&csi_22pins_connector CSI_22PINS_IO0 GPIO_ACTIVE_LOW>;

port {
imx335_ep_out: endpoint {
remote-endpoint-label = "csi_22pins_ep_in";
bus-type = <VIDEO_BUS_TYPE_CSI2_DPHY>;
data-lanes = <1 2>;
};
};
};
};

&csi_22pins_connector {
/* Power the camera module */
en-module-gpios {
gpio-hog;
gpios = <CSI_22PINS_IO1 GPIO_ACTIVE_HIGH>;
output-high;
};
};
1 change: 1 addition & 0 deletions drivers/video/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,5 +21,6 @@ zephyr_library_sources_ifdef(CONFIG_VIDEO_ESP32 video_esp32_dvp.c)
zephyr_library_sources_ifdef(CONFIG_VIDEO_MCUX_SDMA video_mcux_smartdma.c)
zephyr_library_sources_ifdef(CONFIG_VIDEO_EMUL_IMAGER video_emul_imager.c)
zephyr_library_sources_ifdef(CONFIG_VIDEO_EMUL_RX video_emul_rx.c)
zephyr_library_sources_ifdef(CONFIG_VIDEO_IMX335 imx335.c)

zephyr_linker_sources(DATA_SECTIONS video.ld)
2 changes: 2 additions & 0 deletions drivers/video/Kconfig
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -88,4 +88,6 @@ source "drivers/video/Kconfig.emul_imager"

source "drivers/video/Kconfig.emul_rx"

source "drivers/video/Kconfig.imx335"

endif # VIDEO
12 changes: 12 additions & 0 deletions drivers/video/Kconfig.imx335
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
# IMX335

# Copyright (c) 2025 STMicroelectronics.
# SPDX-License-Identifier: Apache-2.0

config VIDEO_IMX335
bool "IMX335 CMOS digital image sensor"
select I2C
depends on DT_HAS_SONY_IMX335_ENABLED
default y
help
Enable driver for IMX335 CMOS digital image sensor device
Loading