一句话：提出 COMMITUP，用 LLM 以“理解变更—检索相似示例模仿—执行更新”三步自动更新方法级过时测试代码。
软件测试保障了持续演化系统的可靠性，但现代快速迭代开发中测试套件常滞后于生产代码，产生大量过时测试。已有工作借助大语言模型自动更新过时测试代码并取得领先效果。本文提出 COMMITUP，一种模拟人类修复思路的方法级过时测试代码自动更新技术：首先理解生产代码的修改（comprehend），然后检索相似历史/示例进行模仿（imitate），最后执行测试代码更新（update）。作者在从真实 Java 项目整理的数据集上评估 COMMITUP，生成的更新分别达到 96.4%、94.4% 和 93.1% 的可编译、无运行时失败与全覆盖成功率，展示了优于基线的性能，为基于 LLM 的测试代码更新提供了新思路。数据集与代码已开源。
Software testing plays a crucial role in software engineering, ensuring the reliability and correctness of evolving systems. Well-maintained test suites are essential for ensuring software quality. However, in modern development cycles that emphasize rapid feature iteration, the co-evolution of test suites often lags behind, leading to more appearance of obsolete tests. To this end, automated approaches for updating obsolete test code have been proposed, and recent approaches have achieved the state-of-the-art performance with the support of large language models (LLMs). This paper presents COMMITUP, a new approach that leverages LLMs to effectively automate method-level obsolete test code updates. COMMITUP mimics how humans solve the problem, first comprehending the code modifications, searching for similar examples to imitate, and finally performing the update. We evaluate COMMITUP on a curated dataset from real-world Java projects. The results demonstrate the superior performance of COMMITUP, achieving 96.4%, 94.4%, 93.1% success rates for generating compilable, runtime failure-free, and full coverage updates, respectively. We believe our study can provide new insight into LLM-based test code update. The dataset and code are available at https://github.com/SoftWiser-group/CommitUp.
DOI 原文 ·
@inproceedings{paperbot9360,
title = {Comprehend, Imitate, and then Update: Unleashing the Power of LLMs in Test Suite Evolution},
author = {Tangzhi Xu and Jianhan Liu and Yuan Yao and Cong Li and Feng Xu and Xiaoxing Ma},
booktitle = {ASE},
year = {2025},
doi = {10.1109/ase63991.2025.00046}
}