一句话：提出 ProphetAgent，从自然语言测试用例自动合成可执行 GUI 测试脚本，成功率 78.1%。
移动应用 GUI 测试对保证软件质量与用户满意度至关重要，而企业常维护大量自然语言编写的测试用例，需人工转换为可执行脚本以进行回归和兼容性测试；在需求变更与版本更新频繁、用例规模庞大的情况下，这一过程成本高昂。本文提出 ProphetAgent，能够从自然语言测试用例自动合成可执行 GUI 测试。该方法首先构建带有语义信息的聚类 UI 转移图（CUTG），随后利用大语言模型基于 CUTG 和自然语言测试用例生成可执行测试。在抖音、豆包及 6 个开源应用共 120 条测试用例上，ProphetAgent 达到 78.1% 的成功率，优于 AppAgent（21.4%）和 AutoDroid（32.5%）。字节跳动测试平台的统计数据还显示，ProphetAgent 使测试人员合成 UI 测试的效率提升了 260%。
GUI tests is crucial for ensuring software quality and user satisfaction of mobile apps. In practice, companies often maintain extensive test cases written in natural language. Testers need to convert these test cases into executable scripts for regression and compatibility testing. Requirement changes or version updates often necessitate the addition and modification to these test cases. Thus, when faced with large volumes of test cases and regular updates, this process becomes costly, which is a common challenge across the industry. To address this issue, this paper proposes ProphetAgent that can automatically synthesize executable GUI tests from the test cases written in natural language. ProphetAgent first constructs a Clustered UI Transition Graph (CUTG) enriched with semantic information, then leverages large language models to generate the executable test case based on CUTG and test cases written in natural language. Experiment results show that ProphetAgent achieved a 78.1% success rate across 120 test cases in Douyin, Doubao, and six open-source apps, surpassing existing automated approaches (21.4% for AppAgent and 32.5% for AutoDroid). Additionally, statistical data from ByteDance's testing platform show that ProphetAgent increased testers' efficiency in synthesizing UI tests by 260%.
DOI 原文 ·
@inproceedings{paperbot8979,
title = {ProphetAgent: Automatically Synthesizing GUI Tests from Test Cases in Natural Language for Mobile Apps},
author = {Qichao Kong and Zhengwei Lv and Yiheng Xiong and Dingchun Wang and Jingling Sun and Ting Su and Letao Li and Xu Yang and Gang Huo},
booktitle = {FSE},
year = {2025},
doi = {10.1145/3696630.3728543}
}