2D to 3D Image Reconstruction

A comprehensive computer vision project that evaluates feature detection algorithms and implements 3D reconstruction from 2D medical images using point cloud generation techniques.

🎯 Project Overview

This project focuses on comparing and evaluating different feature detection and matching algorithms (SIFT, SURF, ORB, and AKAZE) for medical image processing, specifically on Pancreas CT datasets. The goal is to determine the most efficient algorithm for feature matching and utilize the results for 3D image reconstruction using point cloud techniques.

✨ Key Features

• Multi-Algorithm Feature Detection: Implements SIFT, SURF, ORB, and AKAZE algorithms
• RANSAC Outlier Elimination: Improves feature matching accuracy by over 90%
• Performance Analysis: Comprehensive evaluation of computation time and matching efficiency
• 3D Reconstruction: Generates 3D point clouds from 2D medical images
• Ensemble Methods: Combines multiple algorithms for improved results
• Medical Image Support: Handles DICOM format images

🏗️ Project Structure

2D_to_3D_image_reconstruction/
├── Single.py                    # Individual algorithm evaluation and comparison
├── 3Dconst_FM.py              # 3D reconstruction using feature matching
├── Combineafterransac.py      # Ensemble algorithm combination with RANSAC
├── Report_IEEE.pdf            # Detailed project report and analysis
└── README.md                  # This file

🔧 Requirements

Dependencies

• OpenCV (cv2)
• NumPy
• PyDICOM
• scikit-image
• matplotlib
• pandas
• itertools

Installation

pip install opencv-python numpy pydicom scikit-image matplotlib pandas

📁 File Descriptions

1. Single.py - Algorithm Evaluation

Purpose: Evaluates individual feature detection algorithms and compares their performance.

Key Features:

• Loads DICOM images from Pancreas CT dataset
• Implements SIFT, SURF, ORB, and AKAZE algorithms
• Applies RANSAC for outlier elimination
• Measures computation time and matching accuracy
• Generates performance metrics for each algorithm

Usage:

python Single.py

Output: Performance statistics for each algorithm including:

• Number of keypoints detected
• Feature matching results
• RANSAC-filtered matches
• Computation and matching times

2. 3Dconst_FM.py - 3D Reconstruction

Purpose: Creates 3D point clouds from 2D medical images using feature matching.

Key Features:

• Processes sequential DICOM slices
• Detects features between consecutive images
• Generates 3D coordinates with depth information
• Exports results as PLY point cloud files
• Configurable slice thickness and scaling

Usage:

python 3Dconst_FM.py

Configuration:

# Adjustable parameters
slice_thickness = 1.5  # Distance between slices
xy_scale = 1          # XY distance scaling
algo = 'AKAZE'        # Feature detection algorithm

Output: PLY format point cloud files (e.g., test_AKAZE.ply)

3. Combineafterransac.py - Ensemble Methods

Purpose: Combines multiple feature detection algorithms for improved results.

Key Features:

• Tests algorithm combinations systematically
• Applies RANSAC filtering to combined results
• Evaluates ensemble performance metrics
• Generates comprehensive comparison data

Usage:

python Combineafterransac.py

🚀 Getting Started

1. Setup Environment

# Clone the repository
git clone <repository-url>
cd 2D_to_3D_image_reconstruction

# Install dependencies
pip install -r requirements.txt

2. Prepare Data

• Place DICOM images in the appropriate directory structure
• Ensure images are named sequentially (e.g., 1-001.dcm, 1-002.dcm)
• Update file paths in the scripts if necessary

3. Run Analysis

# Evaluate individual algorithms
python Single.py

# Generate 3D reconstruction
python 3Dconst_FM.py

# Test ensemble methods
python Combineafterransac.py

📊 Algorithm Performance

The project evaluates four major feature detection algorithms (SIFT, SURF, ORB, and AKAZE) and measures the following metrics for each:

Measured Performance Metrics:

• KP1: Number of keypoints detected in the first image
• KP2: Number of keypoints detected in the second image
• Matches: Number of feature matches before RANSAC filtering
• Matches after RANSAC: Number of feature matches after outlier elimination
• Computation Time: Time taken for feature detection and extraction (in milliseconds)
• Matching Time: Time taken for feature matching between images (in milliseconds)

Key Findings from Analysis:

• ORB Efficiency: ORB was determined to be the most efficient algorithm for matching inner features
• RANSAC Impact: RANSAC algorithm improved feature matching accuracy by over 90%
• Ensemble Benefits: Combining multiple algorithms provided better results than individual approaches

Note: Specific performance values are generated when running the analysis scripts and depend on the input dataset.

🎯 Key Findings

1. ORB Efficiency: ORB was determined to be the most efficient algorithm for matching inner features
2. RANSAC Impact: RANSAC algorithm improved feature matching accuracy by over 90%
3. Ensemble Benefits: Combining multiple algorithms provided better results than individual approaches
4. Medical Image Suitability: The methods are particularly effective for CT scan datasets

🔬 Technical Details

Feature Detection

• SIFT: Scale-Invariant Feature Transform
• SURF: Speeded Up Robust Features
• ORB: Oriented FAST and Rotated BRIEF
• AKAZE: Accelerated-KAZE

RANSAC Implementation

• Model: AffineTransform
• Min Samples: 4
• Residual Threshold: 8
• Max Trials: 10,000

3D Reconstruction

• Depth Calculation: Based on slice thickness
• Coordinate System: 3D point cloud (X, Y, Z)
• Output Format: PLY (Polygon File Format)

📈 Results and Output

Performance Metrics

• Keypoint detection count
• Feature matching accuracy
• RANSAC filtering results
• Computation time analysis
• Memory usage statistics

3D Output

• Point cloud files (.ply format)
• 3D visualization capabilities
• Depth information preservation
• Scalable coordinate system