一句话：提出 AgoneTest，可规模化地用 LLM 自动生成并评估类级单元测试套件，并发布新数据集与质量评估方法。
单元测试对软件正确性至关重要，但编写测试耗时且成本高昂，自动化需求迫切。利用大语言模型（LLM）生成单元测试是有前景的方案，然而已有研究多集中于简单、小规模场景，对真实应用中大规模集成与评估有效性的理解仍存在空白。本文提出 AgoneTest，一个面向复杂类级测试套件自动生成与评估的系统。其贡献包括：可扩展的自动化系统、用于严格评估的新构建数据集，以及一套细致的测试质量评估方法。该系统能够自动为类生成测试并对其质量加以评价，从而在真实规模场景下考察 LLM 所生成测试的集成效果与评估效能。
Software correctness is crucial, with unit testing playing an indispensable role in the software development lifecycle. However, creating unit tests is time-consuming and costly, underlining the need for automation. Leveraging Large Language Models (LLMs) for unit test generation is a promising solution, but existing studies focus on simple, small-scale scenarios, leaving a gap in understanding LLMs’ performance in real-world applications, particularly regarding integration and assessment efficacy at scale. Here, we present AgoneTest, a system focused on automatically generating and evaluating complex class-level test suites. Our contributions include a scalable automated system, a newly developed dataset for rigorous evaluation, and a detailed methodology for test quality assessment.CCSCONCEPTS• Software and its engineering → Automatic programming; Software testing and debugging.
@inproceedings{LopsNRT24,
title = {AgoneTest: Automated creation and assessment of Unit tests leveraging Large Language Models},
author = {Andrea Lops and Fedelucio Narducci and Azzurra Ragone and Michelantonio Trizio},
booktitle = {ASE},
year = {2024},
doi = {10.1145/3691620.3695318}
}