一句话：提出首个生成非最优决策场景的方法 Decictor，用于评估自动驾驶路径规划决策鲁棒性。
当前自动驾驶系统测试主要关注安全，非安全关键性能尤其是路径规划决策（PPD）鲁棒性评估不足。所谓鲁棒性是 ADS 在环境发生细微变化后能否仍保持最优 PPD，但缺少评估 PPD 最优性的明确 oracle，且难以搜索导致非最优 PPD 的场景。本文提出首个生成非最优决策场景（NoDS）的方法 Decictor，由非侵入式变异、一致性检查与反馈三部分组成。非侵入式变异进行保守修改以在变异场景中保留原最优路径；一致性检查通过比较原场景与变异场景的驾驶路径判断是否存在非最优 PPD；反馈指标融合 AV 运动的时空维度以引导 NoDS 生成。在开源生产级 ADS 百度 Apollo 上，Decictor 共生成 63.9 个 NoDSs，最佳基线仅检测到 35.4 个，验证其有效性。
Autonomous Driving System (ADS) testing is crucial in ADS development, with the current primary focus being on safety. However, the evaluation of non-safety-critical performance, particularly the ADS's ability to make optimal decisions and produce optimal paths for autonomous vehicles (AVs), is also vital to ensure the intelligence and reduce risks of AVs. Currently, there is little work dedicated to assessing the robustness of ADSs' path-planning decisions (PPDs), i.e., whether an ADS can maintain the optimal PPD after an insignificant change in the environment. The key challenges include the lack of clear oracles for assessing PPD optimality and the difficulty in searching for scenarios that lead to non-optimal PPDs. To fill this gap, in this paper, we focus on evaluating the robustness of ADSs' PPDs and propose the first method, Decictor, for generating nonoptimal decision scenarios (NoDSs), where the ADS does not plan optimal paths for AVs. Decictor comprises three main components: Non-invasive Mutation, Consistency Check, and Feedback. To overcome the oracle challenge, Non-invasive Mutation is devised to implement conservative modifications, ensuring the preservation of the original optimal path in the mutated scenarios. Subsequently, the Consistency Check is applied to determine the presence of nonoptimal PPDs by comparing the driving paths in the original and mutated scenarios. To deal with the challenge of large environment space, we design Feedback metrics that integrate spatial and temporal dimensions of the AV's movement. These metrics are crucial for effectively steering the generation of NoDSs. Therefore, Decictor can generate NoDSs by generating new scenarios and then identifying NoDSs in the new scenarios. We evaluate Decictor on Baidu Apollo, an open-source and production-grade ADS. The experimental results validate the effectiveness of Decictor in detecting non-optimal PPDs of ADSs. It generates 63.9 NoDSs in total, while the best-performing baseline only detects 35.4 NoDSs.
@inproceedings{paperbot8798,
title = {Decictor: Towards Evaluating the Robustness of Decision-Making in Autonomous Driving Systems},
author = {Mingfei Cheng and Xiaofei Xie and Yuan Zhou and Junjie Wang and Guozhu Meng and Kairui Yang},
booktitle = {ICSE},
year = {2025},
doi = {10.1109/icse55347.2025.00114}
}