一句话：提出MORLOT，用强化学习生成动态环境变化序列并结合多目标搜索，提升DNN使能系统的在线测试效果。
DNN已广泛用于自动驾驶等信息物理系统，确保这类DNN使能系统（DES）的正确行为至关重要。在线测试让系统与环境闭环交互，但环境变量（如光照）在测试执行中通常保持不变，原因是可探索的场景空间会急剧增大且需同时测试多个需求。本文提出MORLOT，将强化学习与多目标搜索结合用于在线测试：强化学习增量式生成环境变化序列，多目标搜索则决定具体变化，使测试更可能覆盖尚未满足的目标。作者在CARLA高保真模拟器中集成端到端驾驶的DNN使能ADS（Transfuser）进行实证评估，结果表明MORLOT在效果与效率上显著优于对比方法，且效应量大，适合在动态变化环境中考虑多重安全需求来测试DES。
Deep Neural Networks (DNNs) have been widely used to perform real-world tasks in cyber-physical systems such as Autonomous Driving Systems (ADS). Ensuring the correct behavior of such DNN-Enabled Systems (DES) is a crucial topic. Online testing is one of the promising modes for testing such systems with their application environments (simulated or real) in a closed loop, taking into account the continuous interaction between the systems and their environments. However, the environmental variables (e.g., lighting conditions) that might change during the systems' operation in the real world, causing the DES to violate requirements (safety, functional), are often kept constant during the execution of an online test scenario due to the two major challenges: (1) the space of all possible scenarios to explore would become even larger if they changed and (2) there are typically many requirements to test simultaneously. In this paper, we present MORLOT (Many-Objective Rein-forcement Learning for Online Testing), a novel online testing approach to address these challenges by combining Reinforcement Learning (RL) and many-objective search. MORLOT leverages RL to incrementally generate sequences of environmental changes while relying on many-objective search to determine the changes so that they are more likely to achieve any of the uncovered objectives. We empirically evaluate MORLOT using CARLA, a high-fidelity simulator widely used for autonomous driving research, integrated with Transfuser, a DNN-enabled ADS for end-to-end driving. The evaluation results show that MORLOT is significantly more effective and efficient than alternatives with a large effect size. In other words, MORLOT is a good option to test DES with dynamically changing environments while accounting for multiple safety requirements.
DOI 原文 · arXiv · PDF（开放获取） · DBLP
@inproceedings{abs-2210-15432,
title = {Many-Objective Reinforcement Learning for Online Testing of DNN-Enabled Systems},
author = {Fitash Ul Haq and Donghwan Shin and Lionel C. Briand},
booktitle = {ICSE},
year = {2023},
doi = {10.1109/icse48619.2023.00155}
}