Skip to content

Issue with Realsense Config #15

@ShreyasKuntnal

Description

@ShreyasKuntnal

I followed the repository's guidelines but encountered the error shown below.

I am using only the Realsense camera because I currently don't have a 4-camera setup ready. It's indicated that the OmniNXT should work with just the Realsense camera, correct?
This is my issue raised with much more detailed info about the issue and with my PC/Jetson specs.Github Issue Link
Output for roslaunch d2vins realsense.launch.

... logging to /root/.ros/log/4c34f14e-6abb-11ef-a642-847b577263a2/roslaunch-vt008-8464.log
Checking log directory for disk usage. This may take a while.
Press Ctrl-C to interrupt
Done checking log file disk usage. Usage is <1GB.

started roslaunch server http://vt008:37361/

SUMMARY
========

PARAMETERS
 * /d2comm/lcm_uri: udpm://224.0.0.25...
 * /d2comm/self_id: 1
 * /d2pgo/is_4dof: False
 * /d2pgo/self_id: 1
 * /d2pgo/vins_config_path: /root/swarm_ws/sr...
 * /d2vins/accept_min_3d_pts: 30
 * /d2vins/depth_far_thres: 100
 * /d2vins/detector_match_thres: 0.7
 * /d2vins/enable_loop: True
 * /d2vins/enable_network: True
 * /d2vins/enable_pub_remote_img: False
 * /d2vins/enable_sub_remote_img: False
 * /d2vins/init_query_thres: 0.2
 * /d2vins/inter_drone_init_frames: 3
 * /d2vins/is_4dof: False
 * /d2vins/jpg_quality: 75
 * /d2vins/lcm_uri: udpm://224.0.0.25...
 * /d2vins/loop_cov_pos: 0.1
 * /d2vins/lower_cam_as_main: False
 * /d2vins/match_index_dist: 100
 * /d2vins/match_index_dist_remote: 20
 * /d2vins/min_direction_loop: 1
 * /d2vins/min_match_per_dir: 20
 * /d2vins/min_movement_keyframe: 0.1
 * /d2vins/netvlad_model_path: /root/swarm_ws/sr...
 * /d2vins/nonkeyframe_waitsec: 5.0
 * /d2vins/odometry_consistency_threshold: 3.0
 * /d2vins/output_path: /home/dji/output/
 * /d2vins/output_raw_superpoint_desc: False
 * /d2vins/pca_comp_path: /root/swarm_ws/sr...
 * /d2vins/pca_mean_path: /root/swarm_ws/sr...
 * /d2vins/pca_netvlad: /root/swarm_ws/sr...
 * /d2vins/pos_covariance_per_meter: 0.01
 * /d2vins/query_thres: 0.2
 * /d2vins/recv_msg_duration: 0.5
 * /d2vins/self_id: 1
 * /d2vins/send_all_features: False
 * /d2vins/send_img: False
 * /d2vins/send_whole_img_desc: False
 * /d2vins/show: False
 * /d2vins/superglue_model_path: /root/swarm_ws/sr...
 * /d2vins/superpoint_model_path: /root/swarm_ws/sr...
 * /d2vins/superpoint_thres: 0.1
 * /d2vins/triangle_thres: 0.012
 * /d2vins/vins_config_path: /root/swarm_ws/sr...
 * /d2vins/yaw_covariance_per_meter: 0.003
 * /rosdistro: noetic
 * /rosversion: 1.16.0

NODES
  /
    d2comm (d2comm/d2comm_node)
    d2pgo (d2pgo/d2pgo_node)
    d2vins (d2vins/d2vins_node)

auto-starting new master
process[master]: started with pid [8487]
ROS_MASTER_URI=http://localhost:11311

setting /run_id to 4c34f14e-6abb-11ef-a642-847b577263a2
process[rosout-1]: started with pid [8512]
started core service [/rosout]
process[d2vins-2]: started with pid [8519]
process[d2pgo-3]: started with pid [8520]
process[d2comm-4]: started with pid [8521]
D2Comm starting....
PGO Loaded VINS config from /root/swarm_ws/src/D2SLAM/d2vins/../config/realsense_d435/d435_single.yaml
[D2Comm] Try to initialize LCM URI: udpm://224.0.0.251:7667?ttl=1
[D2PGO] In single mode enable_pcm 1 pcm_thres 2.8
[D2PGO] PGOState: is 6dof
[ INFO] [1725453814.574851196]: [D2Comm] Drone 1 ready.
[ INFO] [1725453814.575459550]: [D2Comm] Starting d2comm lcm thread.
[D2PGONode@1] Initialized
Read VINS config from /root/swarm_ws/src/D2SLAM/d2vins/../config/realsense_d435/d435_single.yaml
superpoint_config.max_keypoints: 150
superpoint_config.onnx_path: 
superpoint_config.engine_path: 
superpoint_config.keypoint_threshold: 0.000000
[12:43:34][info][d2frontend_params.cpp,D2FrontendParams,L181] [D2FrontendParams] enable_search_local_aera not found, use default

[12:43:34][info][d2frontend_params.cpp,D2FrontendParams,L232] [D2Frontend] Using lazy broadcast keyframe: false
[12:43:34][info][d2frontend_params.cpp,readCameraConfigs,L344] Read camera from config file
[12:43:34][info][d2frontend_params.cpp,readCameraConfigs,L368] Camera 0: topic: /camera/infra1/image_rect_raw, calib: /root/swarm_ws/src/D2SLAM/d2vins/../config/realsense_d435/left.yaml, T: T [+0.099,+0.020,+0.010] YPR [-89.4,-1.0,-89.4]
[12:43:34][info][d2frontend_params.cpp,readCameraConfigs,L368] Camera 1: topic: /camera/infra2/image_rect_raw, calib: /root/swarm_ws/src/D2SLAM/d2vins/../config/realsense_d435/right.yaml, T: T [+0.099,-0.030,+0.011] YPR [-89.3,-1.0,-89.4]
[12:43:34][info][d2frontend_params.cpp,generateCameraModels,L262] [D2Frontend] Read camera from /root/swarm_ws/src/D2SLAM/d2vins/../config/realsense_d435/left.yaml
[12:43:34][info][d2frontend_params.cpp,generateCameraModels,L262] [D2Frontend] Read camera from /root/swarm_ws/src/D2SLAM/d2vins/../config/realsense_d435/right.yaml
[12:43:34][warning][d2frontend_params.cpp,generateCameraModels,L282] [D2Frontend] No photometric calibration file provided
[12:43:34][info][d2frontend_params.cpp,generateCameraModels,L307] [D2Frontend] Focal length initialize to: 460.0
[12:43:34][info][loop_cam.cpp,LoopCam,L34] Init CNNs using onnx
Trying to init MobileNetVLADONNX@/root/swarm_ws/src/D2SLAM/d2frontend/../models/mobilenetvlad_dyn_size.onnx tensorrt 0 fp16 1 int8 0 pca 1
[D2FrontEnd] Loading PCA for MobileNetVLADONNX: 
d2vins_node: /usr/include/eigen3/Eigen/src/Core/Block.h:120: Eigen::Block<XprType, BlockRows, BlockCols, InnerPanel>::Block(XprType&, Eigen::Index) [with XprType = Eigen::Matrix<float, -1, -1>; int BlockRows = 1; int BlockCols = -1; bool InnerPanel = false; Eigen::Index = long int]: Assertion `(i>=0) && ( ((BlockRows==1) && (BlockCols==XprType::ColsAtCompileTime) && i<xpr.rows()) ||((BlockRows==XprType::RowsAtCompileTime) && (BlockCols==1) && i<xpr.cols()))' failed.
Stack trace (most recent call last):
#11   Object "", at 0xffffffffffffffff, in 
#10   Source "/root/swarm_ws/src/D2SLAM/d2vins/src/d2vins_node.cpp", line 326, in _start [0x5e238ac20e6d]
        323:   d2vins.stopFrontend();
        324:   d2vins.stopAllThread();
        325:   return 0;
      > 326: }
#9    Source "../csu/libc-start.c", line 308, in __libc_start_main [0x72214bb70082]
#8  | Source "/root/swarm_ws/src/D2SLAM/d2vins/src/d2vins_node.cpp", line 317, in D2VINSNode
    |   315:   // spdlog::set_level(spdlog::level::debug);
    |   316: 
    | > 317:   D2VINSNode d2vins(n);
    |   318:   // ros::AsyncSpinner spinner(4);
    |   319:   // spinner.start();
      Source "/root/swarm_ws/src/D2SLAM/d2vins/src/d2vins_node.cpp", line 305, in main [0x5e238ac202a1]
        302:   }
        303: 
        304:  public:
      > 305:   D2VINSNode(ros::NodeHandle &nh) { Init(nh); }
        306: };
        307: 
        308: int main(int argc, char **argv) {
#7    Source "/root/swarm_ws/src/D2SLAM/d2vins/src/d2vins_node.cpp", line 254, in Init [0x5e238ac5cee7]
        251:   }
        252: 
        253:   void Init(ros::NodeHandle &nh) {
      > 254:     D2Frontend::Init(nh);
        255:     initParams(nh);
        256:     estimator = new D2Estimator(params->self_id);
        257:     d2vins_net = new D2VINSNet(estimator, params->lcm_uri);
#6    Source "/root/swarm_ws/src/D2SLAM/d2frontend/src/d2frontend.cpp", line 295, in Init [0x72214e08e8de]
        292:   loop_net =
        293:       new LoopNet(params->_lcm_uri, params->send_img,
        294:                   params->send_whole_img_desc, params->recv_msg_duration);
      > 295:   loop_cam = new LoopCam(*(params->loopcamconfig), nh);
        296:   feature_tracker = new D2FeatureTracker(*(params->ftconfig));
        297:   feature_tracker->cams = loop_cam->cams;
        298:   loop_detector =
#5    Source "/root/swarm_ws/src/D2SLAM/d2frontend/src/loop_cam.cpp", line 38, in LoopCam [0x72214e00220f]
         35:     netvlad_onnx = new MobileNetVLADONNX(
         36:         config.netvlad_model, img_width, img_height, config.cnn_enable_tensorrt,
         37:         config.cnn_enable_tensorrt_fp16, config.cnn_enable_tensorrt_int8,
      >  38:         config.netvlad_int8_calib_table_name);
         39:         
         40:     SuperPoint::SuperPointConfig sp_config = config.superpoint_config;
         41:     superpoint_ptr = std::make_unique<SuperPoint>(sp_config);
#4  | Source "/root/swarm_ws/src/D2SLAM/d2frontend/include/d2frontend/CNN/mobilenetvlad_onnx.h", line 38, in row
    |    36:             auto pca = load_csv_mat_eigen(params->pca_netvlad);
    |    37:             // This first row is the mean
    | >  38:             pca_mean = pca.row(0).transpose();
    |    39:             // // The rest is the components
    |    40:             pca_comp_T = pca.block(1, 0, pca.rows() - 1, pca.cols());
    | Source "/usr/include/eigen3/Eigen/src/plugins/BlockMethods.h", line 861, in Block
    |   859: inline RowXpr row(Index i)
    |   860: {
    | > 861:   return RowXpr(derived(), i);
    |   862: }
      Source "/usr/include/eigen3/Eigen/src/Core/Block.h", line 120, in MobileNetVLADONNX [0x72214e0300dd]
        117:     EIGEN_DEVICE_FUNC
        118:     inline Block(XprType& xpr, Index i) : Impl(xpr,i)
        119:     {
      > 120:       eigen_assert( (i>=0) && (
        121:           ((BlockRows==1) && (BlockCols==XprType::ColsAtCompileTime) && i<xpr.rows())
        122:         ||((BlockRows==XprType::RowsAtCompileTime) && (BlockCols==1) && i<xpr.cols())));
        123:     }
#3    Source "/build/glibc-LcI20x/glibc-2.31/assert/assert.c", line 101, in __assert_fail [0x72214bb7ffd5]
#2    Source "/build/glibc-LcI20x/glibc-2.31/assert/assert.c", line 92, in __assert_fail_base [0x72214bb6e728]
#1    Source "/build/glibc-LcI20x/glibc-2.31/stdlib/abort.c", line 79, in abort [0x72214bb6e858]
#0    Source "../sysdeps/unix/sysv/linux/raise.c", line 51, in raise [0x72214bb8f00b]
Aborted (Signal sent by tkill() 8519 0)
[d2vins-2] process has died [pid 8519, exit code -6, cmd nice --20 /root/swarm_ws/devel/lib/d2vins/d2vins_node __name:=d2vins __log:=/root/.ros/log/4c34f14e-6abb-11ef-a642-847b577263a2/d2vins-2.log].
log file: /root/.ros/log/4c34f14e-6abb-11ef-a642-847b577263a2/d2vins-2*.log
^C[d2comm-4] killing on exit
[d2pgo-3] killing on exit
terminate called without an active exception
[rosout-1] killing on exit
[master] killing on exit
shutting down processing monitor...
... shutting down processing monitor complete
done

How to fix this issue?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions