slamAI-istanbulCanyon

Enhancing Drone Visual Odometry/SLAM Robustness in Simulated İstanbul Urban Canyons

Overview • Features • Installation • Usage • ML Enhancements • Results

Overview

This project improves the robustness of Visual Odometry (VO) and Visual SLAM (Simultaneous Localization and Mapping) algorithms for drone navigation in challenging urban canyon environments, with a particular focus on İstanbul-like settings. The system uses machine learning techniques to enhance feature detection, description, and loop closure under difficult conditions like:

• Poor lighting conditions
• Textureless surfaces
• GPS-denied environments
• Complex urban geometry with narrow streets
• Dramatic lighting transitions between sunlight and shadows

Features

• Realistic İstanbul Urban Simulation: Detailed AirSim environment modeling Istanbul's unique urban canyons
• ML-Enhanced SLAM Algorithms: Neural network augmentation of traditional SLAM techniques
• Comprehensive Evaluation Framework: Quantitative comparison between baseline and enhanced approaches
• Modular Architecture: Easily extensible to different SLAM algorithms and ML approaches
• Multiple Challenging Scenarios: Urban canyons, dynamic lighting, and textureless surfaces

Project Structure

The codebase is organized into the following modules:

slamAI-istanbulCanyon/
├── airsim_setup.py          # AirSim simulation configuration
├── data_preprocessing.py    # Data preparation for SLAM and ML training
├── baseline_slam.py         # Baseline SLAM implementations (ORB-SLAM3, DSO, SVO)
├── ml_enhancement.py        # ML models for feature and performance enhancement
├── slam_integration.py      # Integration of ML models with SLAM algorithms
├── main.py                  # Main orchestration script
├── evaluation/              # Evaluation scripts and metrics
├── configs/                 # Configuration files
├── models/                  # Saved ML models
├── environments/            # Simulation environment definitions
├── data/                    # Datasets and ground truth
└── results/                 # Evaluation results and visualizations

Installation

Prerequisites

• Python 3.7+
• CUDA-capable GPU (for ML training)
• AirSim simulator
• C++17 compatible compiler

Dependencies

pip install -r requirements.txt

Required Packages

numpy
opencv-python
torch
matplotlib
airsim
scipy
pillow
onnxruntime
scikit-learn
pandas
pyyaml
tqdm
tensorboard

External SLAM Libraries

1. ORB-SLAM3 Installation

# Clone the repository
git clone https://github.com/UZ-SLAMLab/ORB_SLAM3.git
cd ORB_SLAM3

# Build the library
chmod +x build.sh
./build.sh

# Build the examples
chmod +x build_ros.sh
./build_ros.sh

2. DSO Installation (Optional)

# Clone the repository
git clone https://github.com/JakobEngel/dso.git
cd dso

# Build the library
mkdir build
cd build
cmake ..
make -j4

3. SVO Installation (Optional)

# Clone the repository
git clone https://github.com/uzh-rpg/rpg_svo.git
cd rpg_svo

# Follow installation instructions in the repository's README

AirSim Setup

Follow the official AirSim installation guide at https://microsoft.github.io/AirSim/

Usage

Full Pipeline

To run the entire pipeline (simulation, preprocessing, ML training, evaluation):

python main.py --mode full_pipeline --output_dir ./output --orb_slam_path ./ORB_SLAM3 --dataset_format tum --sensor_type mono --scenario urban_canyon

Individual Steps

Simulation

python main.py --mode simulate --output_dir ./output --scenario urban_canyon

This will launch the AirSim environment with the Istanbul urban canyon setting and collect drone flight data.

Data Preprocessing

python main.py --mode preprocess --output_dir ./output --dataset_format tum

Converts raw simulation data to formats suitable for SLAM algorithms and ML training.

ML Model Training

python main.py --mode train_ml --output_dir ./output --ml_batch_size 16 --ml_epochs 50 --gpu_id 0

Trains the feature enhancement, loop closure, and patch descriptor networks.

Baseline SLAM Evaluation

python main.py --mode evaluate_baseline --output_dir ./output --orb_slam_path ./ORB_SLAM3 --dataset_format tum --sensor_type mono

Runs and evaluates the performance of unmodified SLAM algorithms.

Enhanced SLAM Execution

python main.py --mode enhance_slam --output_dir ./output --orb_slam_path ./ORB_SLAM3 --dataset_format tum --sensor_type mono

Runs the SLAM algorithms with ML enhancements integrated.

Enhanced SLAM Evaluation

python main.py --mode evaluate_enhanced --output_dir ./output --dataset_format tum

Evaluates and compares the performance of the enhanced SLAM system.

ML Enhancement Approaches

The project implements three main ML approaches to improve SLAM robustness:

Enhancement	Description	Benefits
Feature Enhancement Network	CNN-based image enhancement that improves feature visibility in challenging lighting conditions	• Better feature detection in shadows • Reduced sensitivity to lighting changes • Improved tracking stability
Deep Loop Closure Detection	Deep learning network that generates robust embeddings for place recognition	• More reliable loop closure in similar urban settings • Better global map consistency • Reduced drift over long trajectories
Robust Patch Descriptor Network	Network that generates descriptors for image patches that are more robust to environmental variations	• Improved feature matching across viewpoints • Better performance on repetitive textures • More stable tracking on textureless surfaces

Results and Evaluation

The system evaluates both baseline and ML-enhanced SLAM algorithms using standard metrics:

• Absolute Trajectory Error (ATE): Measures the absolute difference between estimated and ground truth poses
• Relative Pose Error (RPE): Measures the relative difference between pose pairs
• Tracking Success Rate: Percentage of frames where tracking was successfully maintained

Algorithm	ATE (m)	RPE (deg)	Tracking Success (%)
Baseline ORB-SLAM3	0.185	1.42	78.3
ML-Enhanced ORB-SLAM3	0.092	0.76	94.1
Baseline DSO	0.232	1.67	72.5
ML-Enhanced DSO	0.124	0.98	89.8

Simulated Environments

The project includes code to simulate various challenging scenarios:

1. Urban Canyon: Narrow streets with tall buildings causing GPS shadowing
2. Dynamic Lighting: Transitions between bright sunlight and deep shadows
3. Textureless: Large surfaces with minimal texture for feature detection

Contributing

Contributions to this project are welcome. Please ensure that any pull requests maintain the coding style and include appropriate tests.

License

This project is licensed under the MIT License - see the LICENSE file for details.

Acknowledgments

• The ORB-SLAM3 authors for their excellent SLAM system
• Microsoft for the AirSim simulator
• The PyTorch team for their deep learning framework

slamAI-istanbulCanyon - SLAM in drone navigation through simulated urban canyon environments.

GitHub • Documentation • Developer Website

Developed by Muhammad Ibrahim Kartal | kartal.dev