一句话：提出基于驾驶场景特征的测试约简与优先级排序方法，在削减34%以上回放数据的同时保持故障检测能力。
自动驾驶系统开发中常需回放历史驾驶记录以检验新改动，但记录中场景高度冗余（如在高速上长时间保持同一车道）。本文针对多模块自动驾驶系统提出一种测试约简与优先级排序方法：首先依据驾驶场景模式将记录中的帧编码为特征向量，再按相邻向量的相似度将记录切分为片段，对过长片段进行截断，并删除具有相同向量的冗余片段；剩余片段依据场景覆盖率与稀有度进行优先级排序。作者在工业级多模块自动驾驶系统 Apollo 上实现该方法，并在三种道路地图的多种回归设置下评估。结果显示，该方法在保持相当测试有效性的前提下将原始记录缩减逾 34%，几乎检出全部注入故障；相比三个基线，优先级排序方法在 APFD（平均故障检测百分比）上提升约 22%–39%，在 TOP-K 指标上提升约 41%–53%。
When developing autonomous driving systems (ADS), developers often need to replay previously collected driving recordings to check the correctness of newly introduced changes to the system. However, simply replaying the entire recording is not necessary given the high redundancy of driving scenes in a recording (e.g., keeping the same lane for 10 minutes on a highway). In this pa- per, we propose a novel test reduction and prioritization approach for multi-module ADS. First, our approach automatically encodes frames in a driving recording to feature vectors based on a driving scene schema. Then, the given recording is sliced into segments based on the similarity of consecutive vectors. Lengthy segments are truncated to reduce the length of a recording and redundant segments with the same vector are removed. The remaining seg- ments are prioritized based on both the coverage and the rarity of driving scenes. We implemented this approach on an industry- level, multi-module ADS called Apollo and evaluated it on three road maps in various regression settings. The results show that our approach significantly reduced the original recordings by over 34% while keeping comparable test effectiveness, identifying almost all injected faults. Furthermore, our test prioritization method achieves about 22% to 39% and 41% to 53% improvements over three baselines in terms of both the average percentage of faults detected (APFD) and TOP-K.
DOI 原文 ·
@inproceedings{paperbot7023,
title = {Scenario-based test reduction and prioritization for multi-module autonomous driving systems},
author = {Yao Deng and Xi Zheng and Mengshi Zhang and Guannan Lou and Tianyi Zhang},
booktitle = {FSE},
year = {2022},
doi = {10.1145/3540250.3549152}
}