LIGHTWEIGHT ULTRA-ADVANCED MULTI-SENSOR INTELLIGENT NOISE-ABRIDGE THREAT OVERCOMING RADAR
MICROPHONE ARRAY-BASED DIRECTION OF ARRIVAL OF GUNSHOT DETECTION
It's important to know the precise location of a gunshot — or at least as close as humanly possible — because gun violence isn't going away anytime soon.
Our goal is to harness any available technology or assisting capability to ensure the best possible outcome in worst-case scenarios.
An advanced gunshot detection and classification system using a hybrid approach of:
- Microphone arrays
- FPGA-based signal processing
- Deep learning models
By leveraging CNN-RNN neural architectures and Transformer layers, we enable:
- Real-time gunshot identification
- 3D localization
- Type classification
- Instant feedback with highly accurate 3D localization using TDOA
- Multi-classification approach to detect and identify firearm types in real-world scenarios
- Distinguishes gunshots from other loud noises or distractions
- Reduces false alarms
- Ensures fast & accurate responses
- Immediate detection & 3D localization
- FPGA-based noise filtering and feature extraction
- Deep learning-based classification
- Real-time responsiveness
- Hybridization: Combining CNNs and RNNs for enhanced audio analysis
- Transformer Integration: Improved attention to relevant features
- Real-time Application: Live analysis of hazardous sound events
- Python (ML Models)
- Verilog (FPGA Processing)
- 4 or 6 omnidirectional microphones
- FPGA for real-time signal processing
- GPU/TPU for model training
- LCD Display for output
- TensorFlow
- PyTorch
- Scikit-Learn
- Pandas
- TQDM
- CNN
- RNN (LSTM)
- Regression (TDOA)
- CNN + Fully Connected
- CST Transformer
- Microphone Array captures the sound
- ADC converts it to digital signal
- FPGA applies bandpass filtering (up to 3kHz) & extracts features
- CNN-RNN Hybrid Model processes features
- CST Transformer Layer applies 3 attention types:
- Channel-MHSA: Spatial attention
- Spectral-MHSA: Frequency-based attention
- Temporal-MHSA: Time-based evolution attention
- GELU Activation enhances model performance
- Real-time output shown on display
- Built on existing technologies like FPGAs and Neural Networks
- Uses real-time signal processing and deep learning
- Feasible and scalable with current hardware
- Extract local patterns and hierarchies (e.g., sound textures)
- Reduce dimensionality while preserving important spatial info
- Retain temporal memory
- Model long sequences (like varying gunshot durations)
- LSTM or GRU helps with temporal context in audio
| Model | Gunshot Detected (%) | Muzzle Blast Detected (%) | Shockwave Detected (%) | TDOA Accuracy (%) | Gun Type Classification (%) |
|---|---|---|---|---|---|
| CNN-Only | 65 | 83 | 89 | 72 | 80 |
| RNN-Only | 83 | 85 | 94 | 79 | 91 |
| DNN-Only | 82 | 90 | 96 | 77 | 93 |
| CNN+RNN | 92 | 93 | 98 | 82 | 95 |
| CNN+RNN+CST | 96 | 95 | 97 | 89 | 98 |
- Hardware Limitations: FPGAs have finite resources
- Latency: Real-time needs demand high processing speed
- Noise Interference: May affect accuracy
- Data Availability: High-quality gunshot datasets are rare
- Designed efficient Verilog modules
- Implemented parallel processing for real-time handling
- Developed real-time, low-latency algorithms for gunshot detection
- Integrated high-pass/low-pass filters
- Trained ML models with noise-augmented datasets
- FPGA Filters
- Gunshot Detection Thesis - Auraria Library
- TDOA Localization Field Guide
- Multiple Impulse Acoustic Sources - MDPI
- Gunshot Detection Using Accelerometers - PLOS ONE
- Gun Identification using Transformer - Nature Scientific Reports
- Raytheon's Boomerang Acoustic System
- Object Tracking using TDOA - MATLAB
- Gunshot-like Sounds Detection - MDPI
- Postprint PDF - Google Drive
- FireBrick Project - Tufts University
- Indoor Gunshot Notification System - MDPI