一句话：提出用扩散模型生成测试用例并以终止状态新颖性引导搜索，为决策策略测试发现更多样、更具影响力的失败用例。
决策策略构成了自动驾驶、机器人等关键应用的基础，其可靠性至关重要。已有工作针对求解马尔可夫决策过程（MDP）的决策策略开展测试，但由于策略基于深度神经网络且状态空间无限，构建可扩展且有效的测试框架仍是难题。本文提出一个面向决策策略的测试框架：其一，采用基于生成式扩散模型的测试用例生成器，能够方便地适配不同搜索空间，保证测试用例的实用性与有效性；其二，提出基于终止状态新颖性的引导机制，以多样化智能体行为并提升测试效果。作者在自动驾驶、飞机防撞与游戏等五个广泛使用的基准上开展评估，结果表明该方法相比当前最先进技术能识别出更多样、更具影响力的失败触发用例，并且利用所检测到的失败用例修复被测模型时，其鲁棒性提升优于基线方法。
The reliability of decision-making policies is urgently important today as they have established the fundamentals of many critical applications, such as autonomous driving and robotics. To ensure reliability, there have been a number of research efforts on testing decision-making policies that solve Markov decision processes (MDPs). However, due to the deep neural network (DNN)-based inherit and infinite state space, developing scalable and effective testing frameworks for decision-making policies still remains open and challenging. In this paper, we present an effective testing framework for decision-making policies. The framework adopts a generative diffusion model-based test case generator that can easily adapt to different search spaces, ensuring the practicality and validity of test cases. Then, we propose a termination state novelty-based guidance to diversify agent behaviors and improve the test effectiveness. Finally, we evaluate the framework on five widely used benchmarks, including autonomous driving, aircraft collision avoidance, and gaming scenarios. The results demonstrate that our approach identifies more diverse and influential failure-triggering test cases compared to current state-of-the-art techniques. Moreover, we employ the detected failure cases to repair the evaluated models, achieving better robustness enhancement compared to the baseline method.
@inproceedings{LiWZCCZXMZ23,
title = {Generative Model-Based Testing on Decision-Making Policies},
author = {Zhuo Li and Xiongfei Wu and Derui Zhu and Mingfei Cheng and Siyuan Chen and Fuyuan Zhang and Xiaofei Xie and Lei Ma and Jianjun Zhao},
booktitle = {ASE},
year = {2023},
doi = {10.1109/ase56229.2023.00153}
}