PCLA: A framework for testing autonomous agents in the CARLA simulator

PCLA agents:

--- SimLingo (CarLLava), LMDrive, Transfuser, Transfuser++, Interfuser, NEAT, World on Rails (WoR), Learning By Cheating (LBC), Learning from All Vehicles (LAV) ---

---

PCLA (Pretrained CARLA Leaderboard Agents) is a versatile framework that allows you to use and evaluate the autonomous agents from the CARLA Leaderboard independently of its core codebase and put them on your vehicle.

• PCLA provides a clear method to deploy Autonomous Driving Agents (ADAs) onto a vehicle without relying on the Leaderboard codebase.
• Enables easy switching between ADAs without requiring changes to CARLA versions or programming environments.
• Allows you to have multiple vehicles with different autonomous agents (requires high graphical memory).
• Provides the next movement action computed by the chosen agent, which can then be used in any desired application.
• Is fully compatible with the latest version of CARLA and independent of the Leaderboard’s specific CARLA version.
• Includes 23 different high-performing ADAs trained with 31 distinct training seeds.

Paper available at Foundations of Software Engineering

PCLA was tested on Linux Ubuntu 22 and CARLA 0.9.16 Unreal Engine 4.
A video tutorial on how to use PCLA is available below (update will come soon).

Contents

1. Setup
2. Pre-Trained Weights
3. Autonomous Agents
4. How to Use
5. Navigation
6. Sample Code
7. FAQ
8. Citation

Setup

Download and install the CARLA simulator from the official website. Based on your preference, you can either use quick installation or build from source.
Please make sure CUDA and PyTorch are installed.
Tutorial for installing CUDA on ubuntu
Tutorial for PyTorch

Clone the repository and build the conda environment:

git clone https://github.com/MasoudJTehrani/PCLA
cd PCLA
conda env create -f environment.yml
conda activate PCLA

Please make sure to install torch-scatter according to your own CUDA version. You can check your CUDA version using the python cuda.py code.

--If you want to use PCLA for CARLA 0.9.16, you need to install the wheel from the dist folder.

cd dist
python3 -m pip install carla-0.9.16-cp38-cp38-linux_x86_64.whl

Note: Some agents act weird in CARLA 0.9.16

Additional setups for the LMDrive agent:

conda activate PCLA

cd plca_agents/lmdrive/vision_encoder
pip uninstall timm
python setup.py develop

cd ../LAVIS
python setup.py develop

pip uninstall ftfy -y
pip install "ftfy==6.1.1"

Pre-Trained Weights

You can either run this code to download and unzip the weights automatically

python download_weights.py

or manually download the pre-trained weights from Zenodo or directly from here and extract them into the PCLA/agents/ directory.
Ensure that each folder of pre-trained weights is placed directly next to its respective model's folder. The agents folder should look like this.

├── agents
   ├── transfuserpp
   ├── transfuserpp_pretrained
   ├── interfuser
   ├── interfuser_pretrained
   ├── ...

Autonomous Agents

PCLA includes 23 different autonomous agents and 31 distinct training seeds to choose from.

• SimLingo(CarLLava)

  • Contains 1 agent from the leaderboard 2, previously named CarLLava.
    • simlingo_simlingo : The best performing agent, first place at CARLA Leaderboard 2 SENSORS track.
  • Repository

• LMDrive

  • Contains 3 agent from the leaderboard 1.
    • lmdrive_llava : The best performing LMDrive agent.
    • lmdrive_vicuna : The second best performing LMDrive agent.
    • lmdrive_llama : The third best performing LMDrive agent.
  • Repository

• Transfuser

  • Contains 4 agent from the leaderboard 1 transfuser.
    • tf_tf : The main Transfuser agent.
    • tf_ltf : The LatentTF agent.
    • tf_lf : The Late_Fusion agent.
    • tf_gf : The Geometric_Fusion agent.
  • Repository

• Transfuser++

  • Contains 4 different autonomous agents of Transfuser++ with 3 training seeds for each agent.

    • tfpp_l6_# : Best performing Transfuser++ agent. Second place at CARLA Leaderboard 2 SENSORS track(Tuebingen_AI team)
    • tfpp_lav_# : Transfuser++ but it's not trained on Town02 and Town05.
    • tfpp_wp_# : Transfuser++ WP from their paper's appendix.
    • tfpp_aim_# : Reproduction of the AIM method, explained in their paper's appendix.

  • Replace # with the seed number from 0 to 2.

  • Repository

• Learning from All Vehicles

  • Contains 2 autonomous agents. Needs the CARLA to be run with -vulkan.
    • lav_lav : The original LAV agent.
    • lav_fast : The leaderboard submission of LAV. The codes are slightly optimized for leaderboard inference speed with temporal LiDAR scans.
  • Repository

• Learning By Cheating

  • Contains 2 autonomous agents. Needs the CARLA to be run with -vulkan.
    • lbc_nc : Learning By Cheating, the NoCrash model.
    • lbc_lb : Learning By Cheating, the Leaderboard model.
  • Repository

• World on Rails

  • Contains 2 autonomous agents. Needs the CARLA to be run with -vulkan.
    • wor_nc : World on Rails, the NoCrash model.
    • wor_lb : World on Rails, the Leaderboard model.
  • Repository

• NEAT

  • Contains 4 different autonomous agents.
    • neat_neat
    • neat_aimbev
    • neat_aim2dsem
    • neat_aim2ddepth
  • Repository

• Interfuser

  • Contains 1 autonomous agent.
    • if_if : Second best performing CARLA Leaderboard 1 SENSORS track agent.
  • Repository

How to Use

First, run CARLA. You only need -vulkan for LBC, WoR, and LAV agents

./CarlaUE4.sh -vulkan

Then open another terminal and run your code.
To use PCLA, simply import it and use the PCLA class to define an autonomous vehicle with your chosen autonomous agent.

from PCLA import PCLA

agent = "tf_tf"
route = "./sample_route.xml"
pcla = PCLA(agent, vehicle, route, client)

ego_action = pcla.get_action()
vehicle.apply_control(ego_action)

In the code above, the agent is your chosen autonomous agent. You can choose your agent from the list of Autonomous Agents.
You also need to pass the route that you want your vehicle to follow. The route should be in the format of the Leaderboard waypoints as an XML file.
To make it easy, PCLA provides you with a function called routeMaker() that gets an array of CARLA waypoints, reformats it to a Leaderboard format, and save it as an XML file. A tutorial on how to use that is provided in Navigation
The other arguments you have to pass to PCLA are the client and the vehicle you want to put the agent on.
To get one action in a frame from the agent and apply it to your vehicle, you can call the pcla.get_action method.
Example:

ego_action = pcla.get_action()
vehicle.apply_control(ego_action)

Finally, to destroy and cleanup the vehicle, sensors, and the PCLA variables, you can call

pcla.cleanup()

Navigation

You can use PCLA to generate waypoints between two locations or generate routes usable for PCLA. If you want to find locations to navigate your vehicle through the city, you can use the spawn_points.py file to see all the spawn points and their associated number.

python pcla_functions/spawn_points.py

---

You can then use the location_to_waypoint() method to generate waypoints between two CARLA locations. For example:

from PCLA import location_to_waypoint

vehicle_spawn_points = world.get_map().get_spawn_points() # Carla spawn points
startLoc = vehicle_spawn_points[31].location # Start location
endLoc = vehicle_spawn_points[42].location # End location
waypoints = location_to_waypoint(client, startLoc, endLoc) # Returns waypoints between two locations

Then pass the waypoints to route_maker() to make the XML file usable for PCLA.

PCLA.route_maker(waypoints)

---

All together: Example of generating XML route from a list of CARLA waypoints extracted from two CARLA locations:

from PCLA import route_maker
from PCLA import location_to_waypoint

client = carla.Client('localhost', 2000)
world = client.get_world()

vehicle_spawn_points = world.get_map().get_spawn_points() # Carla spawn points
startLoc = vehicle_spawn_points[31].location # Start location
endLoc = vehicle_spawn_points[42].location # End location
waypoints = location_to_waypoint(client, startLoc, endLoc)  # Returns waypoints between two locations
route_maker(waypoints, "route.xml")  # Returns waypoints usable for PCLA

Sample Code

A sample code is provided for you to test PCLA. Just go to the PCLA directory and run:

python sample.py

This sample is in Town02 of the CARLA simulator and uses the LAV agent.

Attention: you may need to change the vehicle spawn point's number on line 43 to something else based on your CARLA version.

FAQ

Frequently asked questions and possible issues are solved in the issues section. If you have a request for a new agent, feel free to ask me.

Citation

If you find PCLA useful, please consider giving it a star 🌟, and cite the published paper.

@inproceedings{tehrani2025pcla,
  title={PCLA: A Framework for Testing Autonomous Agents in the CARLA Simulator},
  author={Tehrani, Masoud Jamshidiyan and Kim, Jinhan and Tonella, Paolo},
  booktitle={Proceedings of the 33rd ACM International Conference on the Foundations of Software Engineering},
  pages={1040--1044},
  year={2025}
}