一句话：研究图像域转换器对自动驾驶 DNN 测试的影响，并提出 SAEVAE 以缩小真实与合成图像分布差距。
自动驾驶系统（ADS）的深度神经网络通常在真实图像上训练，却在模拟器生成的合成图像上测试，训练与测试分布不一致会错误地降低测试准确率。已有工作建议用域到域转换器拉近测试集与训练集，但这可能影响测试的可靠性、有效性和效率。本文研究转换器是否会削弱 ADS-DNN 测试揭示故障的能力、是否带来过多仿真时间开销。作者考虑 CycleGAN、神经风格迁移以及提出的 SAEVAE，在车道保持和目标检测任务上评估。结果表明，转换器显著缩小了分布差异造成的测试准确率差距，SAEVAE 最优；同时不损害测试数据的多样性、覆盖率或故障揭示能力，SAEVAE 的仿真开销可忽略，并能提高离线测试与仿真测试结果的相关性，从而降低仿真测试成本。
Deep Neural Networks (DNNs) for Autonomous Driving Systems (ADS) are typically trained on real-world images and tested using synthetic images from simulators. This approach results in training and test datasets with dissimilar distributions, which can potentially lead to erroneously decreased test accuracy. To address this issue, the literature suggests applying domain-to-domain translators to test datasets to bring them closer to the training datasets. However, translating images used for testing may unpredictably affect the reliability, effectiveness and efficiency of the testing process. Hence, this paper investigates the following questions in the context of ADS: Could translators reduce the effectiveness of images used for ADS-DNN testing and their ability to reveal faults in ADS-DNNs? Can translators result in excessive time overhead during simulation-based testing? To address these questions, we consider three domain-to-domain translators: CycleGAN and neural style transfer, from the literature, and SAEVAE, our proposed translator. Our results for two critical ADS tasks - lane keeping and object detection - indicate that translators significantly narrow the gap in ADS test accuracy caused by distribution dissimilarities between training and test data, with SAEVAE outperforming the other two translators. We show that, based on the recent diversity, coverage, and fault-revealing ability metrics for testing deep-learning systems, translators do not compromise the diversity and the coverage of test data nor do they lead to revealing fewer faults in ADS-DNNs. Further, among the translators considered, SAEVAE incurs a negligible overhead in simulation time and can be efficiently integrated into simulation-based testing. Finally, we show that translators increase the correlation between offline and simulation-based testing results, which can help reduce the cost of simulation-based testing. Our replication package is available online [1].
DOI 原文 ·
@inproceedings{paperbot8283,
title = {Bridging the Gap between Real-world and Synthetic Images for Testing Autonomous Driving Systems},
author = {Mohammad Hossein Amini and Shiva Nejati},
booktitle = {ASE},
year = {2024},
doi = {10.1145/3691620.3695067}
}