Uncertainty Propagation from Sensor Data to Deep Learning Models in Autonomous Driving

Overview

Deep learning has been widely used in Autonomous Driving Systems (ADS). Though significant progress has been made regarding their efficiency and accuracy, uncertainty remains a critical factor affecting ADS safety. We focus on the local path planning task in highway driving scenarios, which is a regression problem, and employ the LSTM-CSP model as the target model. We use Gaussian Processes to quantify the corresponding Aleatoric Uncertainty (AU). For Epistemic Uncertainty (EU), we employ two classic quantification methods, MC Dropout and Deep Ensembles. More importantly, we also studied the cost-effectiveness of MC Dropout and Deep Ensembles in selecting highly-uncertain samples for facilitating model retraining to improve prediction accuracy.

Resource

1. Data: We provide two test sets used in our experiments, including:

• Testset
• Car-following testset

2. Models: We provide 10 pre-trained LSTM-CSP models, including the LSTM-CSP-Optimal model.
3. Pictures: The complete results of the overlap in highly-uncertain samples identified by MC Dropout and Deep Ensembles under different noise levels.

Dependencies

• Support Python 3.5 and above. For a list of dependencies see requirements.txt

  pip install -r requirements.txt

• We recommend using PyCharm software to read and inspect the dataset provided in Data. In the following example, we provide a method to read the file.

  import torch
  import numpy as np
  
  Testset = torch.load("/home/Data/Testset_traj.pt")
  Testset_track = np.load("/home/Data/Testset_tracks.npy", allow_pickle=True)

Usage

1. Download all the files to the local machine.
2. Run the programs in Data Production sequentially to generate the necessary data files.
3. Run the programs in Baseline to quantify the aleatoric and epistemic uncertainties of LSTM-CSP, and calculate the performance and safety metrics.
4. Run the programs in Calculation to calculate the final uncertainty values and calculate the Pearson correlation coefficient between epistemic uncertainty and ADS safety metrics.
5. Run the programs in Analysis sequentially to calculate the overlap and distribution of highly-uncertain samples identified by MC Dropout and Deep Ensembles.