一句话：提出 Cam2Sim，将真实驾驶录像重建为可执行 CARLA 场景，并用高斯泼溅缩小仿真与真实视觉差距。
基于仿真的测试可安全、可重复地评估自动驾驶系统，但合成仿真器输出与真实世界相机观测之间存在差距，限制了测试有效性。Cam2Sim 从真实驾驶录像的相机图像与位姿出发，重建道路几何、自车轨迹、停驻车辆和仿真资产，并引入高斯泼溅渲染，使仿真观测更接近原始录像。工具链支持 ROS 数据提取、停驻车辆检测、基于 OpenStreetMap 的地图生成、CARLA 场景构建、高斯泼溅训练、轨迹回放与闭环测试。作者在真实城市驾驶场景中结合端到端驾驶模型进行验证，评估重建质量、图像生成质量与闭环行为，并与纯仿真基线和真实世界参考对比。结果表明，基于高斯泼溅的渲染缩小了与仿真器渲染的视觉差距，并提升了与真实世界参考的行为相似度。
Simulation-based testing enables safe and repeatable evaluation of autonomous driving systems, but its effectiveness is limited by the gap between synthetic simulator outputs and real-world camera observations. We present Cam2Sim, a tool that transforms real-world driving recordings into executable CARLA scenarios. Starting from camera images and poses, Cam2Sim reconstructs road geometry, ego trajectories, parked vehicles, and simulation assets, and augments the environment with Gaussian Splatting to render camera observations resembling the original recording. The framework supports ROS-based data extraction, parked-vehicle detection, OpenStreetMap-based map generation, CARLA scenario construction, Gaussian Splatting training, trajectory replay, and closed-loop testing. We validate Cam2Sim on a real urban-driving scenario with an end-to-end driving model, evaluating reconstruction quality, image-generation quality, and closed-loop behavior against a simulation-only baseline and the real-world reference. Results show that Gaussian-Splatting-based rendering reduces the visual gap with respect to simulator rendering and improves behavioral similarity to the real-world reference. The artifact is publicly available at https://github.com/ast-fortiss-tum/cam2sim, and a screencast is available at https://youtu.be/KmZ74l1__lI.
@inproceedings{abs-2607-04770,
title = {Cam2Sim: Neural Scenario Reconstruction for Closed-Loop Autonomous Driving Simulation},
author = {Davide Jannussi and Stefano Carlo Lambertenghi and Constantin Carste and Andrea Stocco},
booktitle = {ASE},
year = {2026},
doi = {10.1145/3832783.3834639}
}