一句话：提出MASTest框架，结合多样性与自适应扰动，有效提升多智能体系统测试的故障场景多样性与发现能力。
多智能体系统（MAS）在众多领域成功应用，但其复杂性导致现有单智能体测试方法难以直接适用，且已有对抗攻击研究侧重故障检测却忽视场景多样性。本文提出MASTest测试框架，通过多样性引导探索（涵盖个体多样性和团队多样性）与自适应关键状态利用（设计自适应扰动机制在关键状态扰动动作），以触发更丰富多样的故障场景。在Coop Navi和StarCraft II两个流行MAS模拟环境中评估，与基线相比，故障场景平均距离分别增加29.55%-103.57%和74.07%-370.00%，发现的故障模式分别提高71.44%-300.00%和50%-500.00%。
Multi-agent systems (MASs) have achieved remarkable success in multi-robot control, intelligent transportation, and multiplayer games, etc. Thorough testing for MAS is urgently needed to ensure its robustness in the face of constantly changing and unexpected scenarios. Existing methods mainly focus on single-agent system testing and cannot be directly applied to MAS testing due to the complexity of MAS. To our best knowledge, there are fewer studies on MAS testing. While several studies have focused on adversarial attacks on MASs, they primarily target failure detection from an attack perspective, i.e., discovering failure scenarios, while ignoring the diversity of scenarios. In this paper, to highlight a typical balance between exploration (diversifying behaviors) and exploitation (detecting failures), we propose an advanced testing framework for MAS called with diversity-guided exploration and adaptive critical state exploitation. It incorporates both individual diversity and team diversity, and designs an adaptive perturbation mechanism to perturb the action at the critical states, so as to trigger more and more diverse failure scenarios of the system. We evaluate MASTest on two popular MAS simulation environments: Coop Navi and StarCraft II. Results show that the average distance of the resulting failure scenarios is increased by 29.55%-103.57% and 74.07%-370.00% on two environments compared to the baselines. Also, the failure patterns found by MASTest are improved by 71.44%-300.00% and 50%-500.00% on two experimental environments compared to the baselines.
@inproceedings{paperbot8561,
title = {Enhancing Multi-agent System Testing with Diversity-Guided Exploration and Adaptive Critical State Exploitation},
author = {Xuyan Ma and Yawen Wang and Junjie Wang and Xiaofei Xie and Boyu Wu and Shoubin Li and Fanjiang Xu and Qing Wang},
booktitle = {ISSTA},
year = {2024},
doi = {10.1145/3650212.3680376}
}