一句话：提出 AgoneTest 评估框架与 CLASSES2TEST 数据集，在 Java 中标准化比较 LLM 生成的单元测试与提示策略。
单元测试是软件开发中必要但资源密集的环节。本文提出 AgoneTest——一个针对 LLM 生成 Java 单元测试的自动化评估框架；它并不提出新的测试生成算法，而是为研究者和开发者提供标准化的端到端评估流水线，在贴近真实的条件下比较不同 LLM 与提示策略。作者同时构建了 CLASSES2TEST 数据集，将被测 Java 类与其对应测试类映射起来，并在框架中集成变异得分、测试坏味道等先进度量以进行综合评估。实验结果显示，在能够编译的测试子集上，LLM 生成的测试在覆盖率和缺陷检测方面可媲美甚至超过人工编写的测试；增强的提示策略也有助于提升测试质量。该工作澄清了 LLM 在软件测试中的潜力，并为模型设计、提示工程与测试实践的后续改进提供了洞见。
Unit testing is an essential but resource-intensive step in software development, ensuring individual code units function correctly. This paper introduces AgoneTest, an automated evaluation framework for Large Language Model-generated (LLM) unit tests in Java. AgoneTest does not aim to propose a novel test generation algorithm; rather, it supports researchers and developers in comparing different LLMs and prompting strategies through a standardized end-to-end evaluation pipeline under realistic conditions. We introduce the CLASSES2TEST dataset, which maps Java classes under test to their corresponding test classes, and a framework that integrates advanced evaluation metrics, such as mutation score and test smells, for a comprehensive assessment. Experimental results show that, for the subset of tests that compile, LLM-generated tests can match or exceed human-written tests in terms of coverage and defect detection. Our findings also demonstrate that enhanced prompting strategies contribute to test quality. AgoneTest clarifies the potential of LLMs in software testing and offers insights for future improvements in model design, prompt engineering, and testing practices.
DOI 原文 ·
@inproceedings{paperbot9554,
title = {LLMs for Automated Unit Test Generation and Assessment in Java: The AgoneTest Framework},
author = {Andrea Lops and Fedelucio Narducci and Azzurra Ragone and Michelantonio Trizio and Claudio Bartolini},
booktitle = {ASE},
year = {2025},
doi = {10.1109/ase63991.2025.00198}
}