一句话：提出 Autosimtest，用多个 LLM 智能体协作自动完成 sUAS 仿真测试的场景创建、环境配置、任务生成与结果分析。
小型无人机系统（sUAS）的仿真测试需覆盖恶劣天气、多样地形和不同任务，但场景识别、环境搭建、被测系统集成、任务规划与结果分析仍高度依赖人工，难以穷尽测试。本文提出 Autosimtest：一个由多个 LLM 智能体协作驱动的框架，支持测试场景创建、仿真环境准备、多样化任务生成，以及结果分析与交互式分析界面；其设计可适配多种 sUAS 用例、仿真工具和 SuT 输入要求。作者用基于 PX4 和 ArduPilot 飞控的 SuT 进行仿真测试，分析各智能体性能并收集开发者反馈。结果表明，该框架显著提升测试效率与场景覆盖，减少人工投入并支持更全面的评估。
Thorough simulation testing is crucial for validating the correct behavior of small Uncrewed Aerial Systems (sUAS) across multiple scenarios, including adverse weather conditions (such as wind, and fog), diverse settings (hilly terrain, or urban areas), and varying mission profiles (surveillance, tracking). While various sUAS simulation tools exist to support developers, the entire process of creating, executing, and analyzing simulation tests remains a largely manual and cumbersome task. Developers must identify test scenarios, set up the simulation environment, integrate the System under Test (SuT) with simulation tools, formulate mission plans, and collect and analyze results. These labor-intensive tasks limit the ability of developers to conduct exhaustive testing across a wide range of scenarios. To alleviate this problem, in this paper, we propose Autosimtest, a Large Language Model (LLM)-driven framework, where multiple LLM agents collaborate to support the sUAS simulation testing process. This includes: (1) creating test scenarios that subject the SuT to unique environmental contexts; (2) preparing the simulation environment as per the test scenario; (3) generating diverse sUAS missions for the SuT to execute; and (4) analyzing simulation results and providing an interactive analytics interface. Further, the design of the framework is flexible for creating and testing scenarios for a variety of sUAS use cases, simulation tools, and SuT input requirements. We evaluated our approach by (a) conducting simulation testing of PX4 and ArduPilot flight-controller-based SuTs, (b) analyzing the performance of each agent, and (c) gathering feedback from sUAS developers. Our findings indicate that Autosimtest significantly improves the efficiency and scope of the sUAS testing process, allowing for more comprehensive and varied scenario evaluations while reducing the manual effort.
DOI 原文 ·
@inproceedings{paperbot8919,
title = {LLM-Agents Driven Automated Simulation Testing and Analysis of small Uncrewed Aerial Systems},
author = {Venkata Sai Aswath Duvvuru and Bohan Zhang and Michael Vierhauser and Ankit Agrawal},
booktitle = {ICSE},
year = {2025},
doi = {10.1109/icse55347.2025.00223}
}