一句话：用真实事故报告微调 LLM，并借助碰撞领域特定语言自动诊断仿真测试中安全违规的成因与类别。
仿真测试已被领先企业广泛用于保障自动驾驶系统（ADS）安全，基于场景的测试方法能够生成多样的驾驶场景并发现安全违规。然而目前没有自动化手段判断这些违规是否由被测 ADS 引起、以及属于哪一类别，只能依赖大量人工诊断。为此，作者提出 DiaVio：首先设计一种描述碰撞的领域特定语言（DSL），在自然语言书写的真实事故报告与仿真测试违规场景之间建立对齐；随后用真实事故报告微调基座大语言模型（LLM），使其习得诊断能力，并用微调后的 LLM 对仿真测试中的违规场景进行诊断。评估结果表明，DiaVio 在违规诊断任务上兼具有效性与效率。
Simulation testing has been widely adopted by leading companies to ensure the safety of autonomous driving systems (ADSs). Anumber of scenario-based testing approaches have been developed to generate diverse driving scenarios for simulation testing, and demonstrated to be capable of finding safety violations. However, there is no automated way to diagnose whether these violations are caused by the ADS under test and which category these violations belong to. As a result, great effort is required to manually diagnose violations. To bridge this gap, we propose DiaVio to automatically diagnose safety violations in simulation testing by leveraging large language models (LLMs). It is built on top of a new domain specific language (DSL) of crash to align real-world accident reports described in natural language and violation scenarios in simulation testing. DiaVio fine-tunes a base LLM with real-world accident reports to learn diagnosis capability, and uses the fine-tuned LLM to diagnose violation scenarios in simulation testing. Our evaluation has demonstrated the effectiveness and efficiency of DiaVio in violation diagnosis.
DOI 原文 ·
@inproceedings{paperbot8532,
title = {DiaVio: LLM-Empowered Diagnosis of Safety Violations in ADS Simulation Testing},
author = {You Lu and Yifan Tian and Yuyang Bi and Bihuan Chen and Xin Peng},
booktitle = {ISSTA},
year = {2024},
doi = {10.1145/3650212.3652135}
}