gpu-block-solver and ORBSLAM3

[fede2000rico]

Good evening,
I'm facing problem in compiling this library on my NVIDIA Jetson AGX Orin.

It is possible to upload the procedure (list of operations/commands) to build the ORBSLAM3 with the gpu-block-solver upgrade?

Thank you in advance!

[hvsg]

Hi,

The integration is somewhat tricky. I've added instructions here which I have tested on the Xavier NX. It assumes that your build environment is already set up for both libraries. Please give it a try and let me know if you have any questions or encounter more errors.

[fede2000rico]

Hi thank you very much.
The instructions work also on NVIDIA Jetson AGX Orin Dev Kit 64GB, I just encountered some issues, here is the feedback:

1. Testing the EuRoc dataset, visual datasets work fine (mono & stereo); with the visual+inertial dataset (monoi & stereoi) I get this error:
   There are 1 cameras in the atlas
   Camera 0 is pinhole
   not IMU meas
   not IMU meas
   not enough acceleration
   not enough acceleration
   First KF:0; Map init KF:0
   New Map created with 533 points
   terminate called after throwing an instance of 'std::runtime_error'
   what(): BlockSolver2: Landmark or pose size was 0!
   Aborted (core dumped)

2. Which line of code did you modify in Using the block solver inside Optimizer.cc? Can be this the source of the previous error?

3. Thererefore I want to ask you how can I see that the integration uses correctly the new BA? ie: Testing the times of execution and compare original ORBSLAM3 and ORBSLAM3 with the optimized BA.

Thank you very much or your help and have a nice day.

[hvsg]

Thanks for the feedback.

1. This kind of error can happen if the optimization doesn't use marginalized landmarks (i.e. Schur elimination).
2. Those modifications were done inside Optimizer::LocalInertialBA.
3. For each block solver, you can try measuring the total time taken inside Optimizer::LocalInertialBA using g2o::get_monotonic_time and average that. For finer grained measurements, there's a built-in way (g2o/core/batch_stats.h). There's also (g2o/gpu/BAStats.hpp) for logging into a CSV-like format, but you would have to add the measurement code yourself.

[fede2000rico]

Hi,
at first thank you very much for your help.

1. I tried to debug the code, but I don’t understand how to fix it. Can you please explain a bit more in detail why "This kind of error can happen if the optimization doesn't use marginalized landmarks (i.e. Schur elimination)."
2. Thank you for the specification
3. Thank you for redirecting me to the scripts

Thank you very much again for your help.

[hvsg]

Sure, regarding 1, that error should occur if the number of landmarks or poses in the BA optimization is 0. Currently BlockSolver2X only handles the case where the bundle adjustment problem is divisible into pose (keyframe poses, inertial states, etc) and landmark (map points) variables, and assumes there are no edges between landmarks. The reason for this is to make the solving more efficient (example in Ceres docs).

With g2o, I think the landmark vertices are designated by calling setMarginalized(true). So I expect this block solver will only work inside specific functions like Optimizer::LocalInertialBA, Optimizer::FullInertialBA, Optimizer::LocalBundleAdjustment, and Optimizer::BundleAdjustment.

I have a different branch which drops this restriction, but I do not recommend using it that way because it means there is no work to offload to the GPU except for the linear solver.

Also regarding integration:

• If the BA workload is too small, and uses a fixed-size block solver (like BlockSolver_6_3 instead of BlockSolverX), I found BlockSolver2X won't give a performance improvement for those scenarios. In general, it should give a better speedup for global problems, when combined with the PCGSolver, or cuSOLVER.

[fede2000rico]

Thank you very much.
To generalize my question, if I want to reproduce your result how did you handle this issue with EuRoc dataset? Or what I need to do to bypass/solve what I have encountered?
My goal is to reproduce your work and obtain the statistics on my NVIDIA Jetson AGX Orin and follow your steps.

I followed your instruction for the integration.

Thank you for your help.

[hvsg]

Sorry, I am not sure - I am not able to reproduce the issue for the MH01 dataset in stereo-inertial mode using the instructions. My only guess is to check that BlockSolver2X is not used in any other function except for Optimizer::LocalInertialBA (roughly between lines 2508 to 2540).