Integration of BEVFormer into the AW perception stack

[ramaseshanms-mcw]

Hello Team,

This is Ram from MulticoreWare,

We are studying the feasibility of porting this BEVFormer repo to ROS, and subsequently to AW perception stack.
We see some potential in refactoring this script into a ROS2 Python Inference Node. If functionality is the principal objective for this model, this should help answer it. But in case, its not a accepted convention inside the stack, we will port everything to Cpp but this exercise does have some potential risks and unknowns at this point. Which one would you prefer?

Looping in @mitsudome-r, @xmfcx, @liuXinGangChina, @Selventhiran-Rengaraj-MCW, @r-abishek

Please share your thoughts on this.
Thanks,
Ram

[amadeuszsz]

Hello @ramaseshanms-mcw

Thanks for you initiative!

As OSS community we really appreciate contribution to our software stack, especially if it brings something new or improve existing capabilities.

Regarding implementation, unfortunately we can't accept Python code for inference. To minimize the risk, you could first develop a standalone, ROS 2 Python package and check the performance. If results will look promising, you can start proceeding with C++ package for Autoware.

[liuXinGangChina]

Hi @ramaseshanms-mcw

Thanks for your interest in contributing to the AWF community.

As @amadeuszsz san said, every contribution will strictly follow the contribution guide( which is very similar to the google code convention).
Please enjoy the challenge and joy that hacking give you
And forget about the risk we are here to help to make sure the final pr works robust and well by reviewing and revising

Happy hacking！

Looking forward to see your master piece！

心刚

[ramaseshanms-mcw]

Feasibility Study Summary: Porting BEVFormer to ROS 2 and C++

We have completed a feasibility study on porting BEVFormer to ROS 2 and C++. This effort aims to directly integrate BEVFormer into Autoware Universe, unlike BEVDet, which is currently integrated as a vendor library. Our goal is a C++ implementation optimized with TensorRT using FP32 and FP16 precisions for improved runtime efficiency and seamless integration within the Autoware ecosystem.

---

📌 Pre-Processing & Post-Processing Components

The major pre-processing and post-processing blocks in BEVFormer are illustrated below:

• ✅ Green Blocks: Can be reused/ported from BEVDet/BEVFusion PRs in Autoware Universe.
• ⚪ White Blocks: Require custom C++/ROS 2 implementations.

---

🔧 Next Steps

• Port BEVFormer small variant, chosen for its balance between performance and accuracy.
• Complete ROS 2 and C++ implementation.
• Enable fp16 inference with TensorRT
• Integrate into Autoware Universe as a native module.
• Evaluate performance on nuScenes and, if applicable, AWSim.

---

📊 Model Comparison

Model	NDS	mAP	Weights
BEVFormer Tiny	0.354	0.252	Download
BEVFormer Small	0.478	0.370	Download
BEVFormer Base	0.517	0.416	Download

For more technical details and architecture, visit the official BEVFormer repository.

---

Stay tuned for further updates as we move toward a robust BEVFormer integration into the Autoware stack.
@Selventhiran-Rengaraj-MCW, @r-abishek, @ramaseshanms-mcw @RahulSundarMCW @naveen-mcw Dhinesh_MCW

Looping in @mitsudome-r, @xmfcx, @liuXinGangChina