Raspberry Pi Camera 2.1 on TX2

[LalitPradhan]

@CospanDesign

I was following your blog http://cospandesign.github.io/linux,tx2,kernel,driver/2017/12/15/tx2-rpi-camera-port.html.

I have an Auvidea J106 board and raspberry pi 2.1 CSI cameras. As per your blog,

1. I replaced the files imx219.c and imx219_mode_tbls.h in Linux_for_Tegra/sources/kernel/kernel-4.4/drivers/media/i2c.
2. Commented out the camera lines in tegra186-quill-p3310-1000-a00-00-base.dts
3. Made all the changes as mentioned in tegra186-my-camera-config-a00.dtsi in both modules and platform.
4. downloaded toolchain and your helper scripts in the right location
5. Made changes as mentioned by running ./build-kernel.sh -m
6. Rebuilt kernel using ./build-kernel.sh -r (I can see built-in.mod.c, built-in.o, i2c-mux-pca954x.o, modules.builtin, modules.order files in Linux_for_Tegra_tx2/kernel/drivers/i2c/muxes)
7. ran ./apply_binaries.sh
8. Flashed it with jetpack by running JetPack-L4T-3.1-linux-x64.run

But I never see 3 extra buses as you have mentioned in /dev. I always have the regular 8 i2c devices. I also never see 3 new /dev/videoX entries.

When I run the following in my TX2
sudo su
cd /sys/class/gpio
echo 461 > export
cd gpio461
echo out > direction
echo 0 > value
echo 1 > value
i2cdetect -y -r 0

I see
0 1 2 3 4 5 6 7 8 9 a b c d e f
00: -- -- -- -- -- -- -- -- -- -- -- -- --
10: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
20: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
30: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
40: -- -- -- -- -- -- -- -- -- -- -- -- UU -- -- --
50: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
60: -- -- -- -- 64 -- -- -- -- -- -- -- -- -- -- --
70: -- -- -- -- -- -- -- —
Which is for PiCamera 2.1

But on running gstreamer commands I get an error for pipeline.

Could you please help me out if I did the correct steps or not?

[GiraffeRage]

I can see my PiCamera but still am unable to get any video feed from it. Have you had any luck yet?

[LalitPradhan]

Hey, Not yet. I got a few patches from Ridgerun and built the Kernel using them but still no luck. I need to add a patch to gstreamer to detect RAW10 instead of RAW8. I will try that and update you.

[LalitPradhan]

P.S. I'm able to see video0 now with that patch.

[LalitPradhan]

@GiraffeRage I've got it to capture the images using raspberry pi finally but at the moment its in RAW format (bayer files). Will check if I can convert it to RGB or not and let you know.

[cospan]

Hi, sorry for the late reply,

I wrote the driver a while ago and it worked for the 24.X version of the TX2 Jetpack but when I tried it on the 3X.X version of Jetpack things didn't work.

If you got a patch from Ridgerun that included the driver then you are in good shape.

The main concern I have is the DTS. My board used GPIOs to multiplex the camera I2C, Essentially changing the single Camera I2C to 3 Camera I2Cs. So when I booted up the kernel there were three extra I2C buses (/dev/i2c7, /dev/i2c8, /dev/i2c9).

I'm not sure how the Auvidea boards were designed, if they used GPIOs to control the multiplexer or if they used an I2C based multiplexer. If so then their DTS file would look different than mine... I just looked over their J106 technical reference, it looks like it may not be multiplexed at all.

You might need to modify the DTS file in order to get the cameras to work correctly.

Specifically where I talk about the DTS in the post:

/ {
  host1x {
    //Make seperate module
    i2c-camera-mux {
      //i2c@3180000
      status = "okay";
      compatible = "i2c-mux-gpio";
      #address-cells = <1>;
      #size-cells = <0>;
      mux-gpios     = < &tegra_aon_gpio   CAM_0_MUX GPIO_ACTIVE_HIGH
                        &tegra_main_gpio  CAM_1_MUX GPIO_ACTIVE_HIGH>;

      i2c-parent = <&cam_i2c>;
      //idle-state = <0>;

      i2c@0 {
      ...

This stuff needs to change, it looks like camera 0-N is on I2C0 so you would need to take out "i2c-camera-mux" section because there is no multiplexer any more, to give an example I'm going to remove all the camera specific stuff but it would go from something like this:

/ {
  host1x {
    //Make seperate module
    i2c-camera-mux {
      //i2c@3180000
      status = "okay";
      compatible = "i2c-mux-gpio";
      #address-cells = <1>;
      #size-cells = <0>;
      mux-gpios     = < &tegra_aon_gpio   CAM_0_MUX GPIO_ACTIVE_HIGH
                        &tegra_main_gpio  CAM_1_MUX GPIO_ACTIVE_HIGH>;

      i2c-parent = <&cam_i2c>;
      //idle-state = <0>;

      i2c@0 {
        ...
        imx219_a@10
        ...
      }
      i2c@1 {
        ...
        imx219_c@10
        ...
      }
      i2c@2{
        ...
        imx219_e@10
      }
  }
}

To something like this

/ {
  host1x {
    //Make seperate module
    i2c@0 {
      ...
      imx219_a@10
      ...
    }
    i2c@1 {
      ...
      imx219_c@10
      ...
    }
    i2c@2{
      ...
      imx219_e@10
    }
  }
}

You will also need to modify the 'tegra-camera-platform' so that the 'proc-device-tree' and 'devname' is pointing to the right place.

so what is now written as this:

/* camera control gpio definitions */
/ {
  tegra-camera-platform {
    ...
    modules{
      ...
      module0 {
        ...
        drivernode0 {
          ...
          devname = "imx219 9-0010";
          proc-device-tree = "/proc/device-tree/host1x/i2c-camera-mux/i2c@0/imx219_a@10";
        };
      };
      module1 {
        ...
        drivernode0 {
          ...
          devname = "imx219 10-0010";
          proc-device-tree = "/proc/device-tree/host1x/i2c-camera-mux/i2c@1/imx219_a@10";
        };
      };
      module2 {
        ...
        drivernode0 {
          ...
          devname = "imx219 11-0010";
          proc-device-tree = "/proc/device-tree/host1x/i2c-camera-mux/i2c@2/imx219_a@10";
        };
      };
    };
  };
};

To something like

/* camera control gpio definitions */
/ {
  tegra-camera-platform {
    ...
    modules{
      ...
      module0 {
        ...
        drivernode0 {
          ...
          devname = "imx219 0-0010";
          proc-device-tree = "/proc/device-tree/host1x/i2c@0/imx219_a@10";
        };
      };
      module1 {
        ...
        drivernode0 {
          ...
          devname = "imx219 6-0010";
          proc-device-tree = "/proc/device-tree/host1x/i2c@6/imx219_c@10";
        };
      };
      module2 {
        ...
        drivernode0 {
          ...
          devname = "imx219 2-0010";
          proc-device-tree = "/proc/device-tree/host1x/ii2c@2/imx219_e@10";
        };
      };
    };
  };
};

@LalitPradhan there is a circuitous way to get the bayer to a more useful TIFF format using this repo

Bayer2RGB

Here is how I used it to convert from bayer to tiff

./bayer2rgb --input=data.raw --output=data.tiff --width=1920 --height=1080 --bpp=16 --first=GBRG --method=BILINEAR --tiff

Then use image magik to convert the tiff to png

convert data.tiff data.png

[GiraffeRage]

Hey @cospan,

Thanks for taking the time to help us out!
I went through and made the changes you mentioned. I attached the dtsi file as it is written for me for the Auvidea J100 (for CSI-A and CSI-B). Does there seem to be anything that is obviously out of place? At this point I am not sure if the driver is working or not and if I may have an issue on the board itself that is unrelated.

tegra186-my-camera-config-a00.txt

[LalitPradhan]

@cospan Thanks for the update. I will try them out later on a different TX2. As of now I have the RPI 2.1 Cameras working with J106 on Jetpack 3.1
I used the patches provided by ridgerun. It didn't work with the default imx219.c and the header files provided in the Jetpack but used the ones provided by you on the blog for RPI cameras. I'm able to take pictures and save it in Bayer/ RAW10 mode.

I had already tried the bayer2rgb link you have provided. I somehow run into bad arguments error when I run in on the TX2. But work fine and converts properly on the 1080Ti host machine. I'm working on two things here:

1. How do i get the bayer2rgb running in TX2 without errors.
2. Capture using OpenCV. I have built the open CV with V4L2 libraries but Its pointing to the default gstreamer installed with Jetpack. I'm not sure how I can make it point to the gstreamer installed using the patch update via cmake. Can you please help me with this? I Installed gstreamer using this link and is installed at a separate place: https://developer.ridgerun.com/wiki/index.php?title=Compile_gstreamer_on_Jetson_TX1_and_TX2
   I have to run the gstreamer from ~/gst_1.8.0/out/bin at present using ./gst-launch-1.0 -v v4l2src device=/dev/video0 ! "video/x-bayer, format=rggb, width=640, height=480" ! bayer2rgb ! videoconvert ! xvimagesink sync=false

@GiraffeRage Ridgerun folks were kind and they gave the patches for free. J106 on TX2 for 6 Sony IMX219 sensor cameras on Jetpack 3.1. You might want to mail them regarding this.

[LalitPradhan]

I solved the bayer2rgb issue on TX2 using #jdthomas/bayer2rgb#1 in the bayer2rgb.c and then doing make

[LalitPradhan]

@cospan , Can you help me with imx219_mode_tbls.h file? I wanted the camera to capture the full resolution of 3280x2464. How do I configure this part:
static imx219_mode_values_t mode_1920x1080_values = {
.gain = 0,
.coarse_time = (1766 - 4),
.frame_length=1766,
};
for the 3280x2464? Where can I find these values as a reference and What other changes do I need to keep in mind with respect to both imx219.c & imx219_mode_tbls.h files for the 3280x2464 resolution?

Thanks.