I am having trouble reproducing your depth pointcloud results.

[akira-akp]

Thank you for your repository!

I have a problem when I run the code.
I think that I I think that by using this repository and this dataset, I can get depth pointcloud and SLAM trajectories like in this video(https://www.youtube.com/watch?v=IOuJ7Y6dpeY).

However, when I run it, I get the following pointcloud results, which are not what I am able to reproduce.

Is this the correct combination of rosbag recording data and calibration data I should be using?
If it is incorrect, I would like to know the correct combination.
Additionally, I would like to know if there are any changes in the source code that need to be made to reproduce the data.

Also, is it correct that the aruducam photos in the rosbag recording data are upside down?

---

branch: pr_fix_main(https://github.com/HKUST-Aerial-Robotics/D2SLAM.git)
model file: hitnet_1x240x320_model_float16_quant_opt.onnx
engine file: hitnet_1x240x320_model_float16_quant_opt.trt

data set DL link: https://hkustconnect-my.sharepoint.com/:f:/g/personal/pliuan_connect_ust_hk/EhrnbPJoptRGvYhsa3P6KUIBscF9NArlttBNUqCaTvzsyw?e=QChj0X
using calibration file: quad_cam_calib-camchain-imucam-7-inch-n3.yaml

rosbag data: quadcam_7inch_n3_2023_1_14/eight_noyaw_1-sync.bag

rosbag data: quadcam_ri_sep/quadwalk_around_2-sync.bag

---

[Peize-Liu]

Hi,
Sorry for the late reply. The image sequence of OmniNxt is different from D2SLAM, which I think is the main reason for your issue. I will provide a new calibration file and OmniNxt dataset this week.

[akira-akp]

Thanks for your reply.
I am very much looking forward and waiting for the new calibration files and OmniNxt dataset to be provided by you.
I would like you to provide them via OneDrive as I cannot download them via Baidu or Dropbox in my environment.

[akira-akp]

Hello. @Peize-Liu
I am looking forward to the new calibration files and OmniNxt dataset.
When will these be available?

[akira-akp]

Hello. @Peize-Liu

I was able to output my expected point cloud using “quadcam_7inch_n3_2023_1_14/eight_noyaw_1-sync.bag” and “quadcam_ri_sep/quadwalk_around_2-sync.bag”.

The reason for the incorrect point cloud output as in my previous post was due to the following items.

• The camera indexes in the calibration file are incorrect.
• The aruducam photos in the rosbag recording data are upside down after rectify processing.

I think that the difference in image sequence between D2SLAM and OmniNxt has affected this part of the process.
I would like to hear your opinion if this idea is correct!

Also, I continue to look forward to receiving new calibration files and OmniNxt datasets from you.

Best regards.

[892692498]

Hello， @Peize-Liu .
I am also looking forward to receiving new calibration files and OmniNxt datasets from you. Thank you very much.
Best regards.

[Melvin-leo]

@akira-akp Did you managed to resolve the upside down issue?

[Melvin-leo]

@akira-akp @Peize-Liu May I know how do you use the oak drivers to load the cam datasets? The output of the datasets is at the rostopic of arducam/compressed, but the quadcam depth uses the output from the oak drivers and I am not sure how are we going to use the oak drivers and the oak drivers docker image.

[akira-akp]

@Melvin-leo

Did you managed to resolve the upside down issue?

I simply used cv::rotate().

May I know how do you use the oak drivers to load the cam datasets?

Sorry, I have about that problem too.
I checked only rosbag play results.
I have not been able to confirm that quadcam-depth-estimation is working on the actual aruducam cameras image.

[Melvin-leo]

Thanks for your reply ! Are the results shown above from the quadcam depth estimation? My quadcam depth estimation is giving my pointclouds that are messy and wrong. Seems to be some issues with transform or planar translation.

I am not sure if I am running the depth estimation wrongly. The d2vins odom is fine though.

[MorningXu]

@Melvin-leo Hello！ I've also encountered the same problem. May I ask if you have made any progress?

[Melvin-leo]

@MorningXu Hi! Sorry I didnt have progress yet, was working on something else. Would you let me know if you have any progress?

[1upsteam1]

@akira-akp May I ask how you managed to get it running? Why can't I see my pointcloud data?