The RL-Agent (red car) learned deep navigation with intelligent lane change

Drive Through Deep - An Intelligent Autonomous agent driving in a dense highway scenario.

A reinforcement learning framework of a self-driving car to learn lateral (lane change decision) and longitudinal control (acceleration/deceleration) while navigating through deep traffic.

Task Objectives:

• Make an autonomous agent learn lane change and longitudinal control to navigate through deep traffic while maintaining better safety and higher traffic flow
• Provide a simple openAI like gym environment to work with reinforcement learning (RL) algorithm for lane change and highway driving scenarios using Simulation of Urban Mobility (SUMO) simulator.

Approach

1. Create a custom gym environment using SUMO
2. Define highway driving and lane-changing problem as Markov decision process (MDP)

Installation

---

Install the Latest version of SUMO

Install custom SUMO Environment

git clone git@github.com:lokesh-c-das/intelligent-self-driving-car.git
cd SUMO-RL-ENVIRONMENT
cd gym_sumo
pip install -e .

Highway driving & lane changing problem formulation

We formulate the high driving and lane changing problem as a Markov decision process (MDP), where

State Space:

The state space contains 19 different states of current environment conditions; from ego vehicle (aka RL agent) and its 3 leaders from inlane, target-left lane, target-right lane, and lanes average speed, and lanes density

Action Space

5 discrete action values that map

• 0 --> Keep current lane and speed
• 1 --> Change lane left
• 2 --> Change lane right
• 3 --> Accelerate (Constant amount)
• 4 --> Decelerate (Constant Amount)

Reward

The reward function consists of an efficiency reward, a lane change penalty reward, and a safety reward

Efficiency Reward:

Safety Reward

Lane Change Penalty

Results

Training Reward and Loss