🚗💥 From Words to Collisions: LLM-Guided Evaluation and Adversarial Generation of Safety-Critical Driving Scenarios

We propose a novel framework that leverages Large Language Models (LLMs) for:

• 🧠 Evaluation: Assessing safety-criticality of driving scenarios with use cases: Scenario evaluation and Safety inference.
• 🛠️ Generation: Adversarially generating safety-critical scenarios with controllable agent trajectories.

---

✨ Highlights

🟢 Original Scenario

🔴 Modified Scenario (Collision)

🔥 Updates

• [July 2025] Paper accepted at IEEE ITSC 2025
• [May 2025] Project repository initialized

---

🗂️ Project Structure

📁 LLM/ – LLM-Based Analysis Scripts

• agent/: Analyze agent-based scenarios
• agent_normal/: Analyze normal agent behaviors
• scenario/: Analyze collision scenarios
• scenario_normal/: Analyze normal driving scenarios

📁 Generation/ – Scenario Generation & Safety Metrics

• BEL_Antwerp-1_14_T-1/: Original normal scenarios
• BEL_Antwerp-1_14_T-1n/: Generated adversarial scenarios
• Metrices/: Safety metric for these two scenarios

📁 Data_collection/ – Trajectory Data Tools

• Trajectory_collection/: Collect vehicle trajectories
• Riskscore_calculation/: Compute risk scores
• Safety_metrics_collection/: Extract safety metrics
• CloesdID_identification/: Identify nearby agents
• generate_timestep_report.py: Generate reports for each timestep

📁 Scenarios/ – Dataset of Driving Scenarios

• normal_scenarios/: 100 normal scenarios (Frenetix planner)
• collision_scenarios/: 100 collision scenarios (Frenetix planner)

📁 Results/ – Analysis & Validation

• LLM/: Results from LLM evaluations
• output_validation/: Validation for collision scenarios
• output_validation_normal/: Validation for normal scenarios

⚙️ Setup & Configuration

✅ Requirements

• Python 3.8+
• CommonRoad
• Frenetix Motion Planner

🔐 API Keys

Create a .env file in the root directory with your API keys: OPENAI_API_KEY=your_openai_key GEMINI_API_KEY=your_gemini_key DEEPSEEK_API_KEY=your_deepseek_key

📖 Citation

If you find this work helpful in your research, please consider citing us: