一句话：提出大规模焦点-测试配对数据集 UniTSyn，显著提升 LLM 的单元测试生成准确率与代码覆盖率。
现有代码大语言模型在生成准确、完整的测试方面表现不佳，因为其训练语料未区分测试代码与其他代码，且将测试与被测函数关联对推断预期行为和验证路径至关重要。本文提出大规模数据集 UniTSyn，利用 Language Server Protocol 收集焦点函数—测试对，无需逐项目执行环境或逐语言启发式规则，从而可扩展地构建数据。UniTSyn 覆盖五种主流编程语言，包含 270 万对焦点-测试。实验基于 UniTSyn 构建自回归 LLM，并在多种语言上评估，结果表明该数据集显著提升模型对单元测试表示的学习与理解，改善测试生成准确率和代码覆盖率。
The remarkable capability of large language models (LLMs) in generating high-quality code has drawn increasing attention in the software testing community. However, existing code LLMs often demonstrate unsatisfactory capabilities in generating accurate, complete tests since they were trained on code snippets collected without differentiating between code for testing and for other purposes. In this paper, we present a large-scale dataset, UniTSyn, which can enhance LLMs for Unit Test Synthesis. Associating tests with the tested functions is crucial for LLMs to infer the expected behavior and the logic paths to be verified. By leveraging Language Server Protocol, UniTSyn achieves the challenging goal of collecting focal-test pairs without per-project execution setups or per-language heuristics, which tend to be fragile and difficult to scale. Containing 2.7 million focal-test pairs across five mainstream programming languages, it can enhance the test generation ability of LLMs. Our experiments demonstrate that, by building an autoregressive LLM based on UniTSyn, we can achieve significant benefits in learning and understanding unit test representations, resulting in improved generation accuracy and code coverage across all the evaluated programming languages.
DOI 原文 ·
@inproceedings{paperbot8498,
title = {UniTSyn: A Large-Scale Dataset Capable of Enhancing the Prowess of Large Language Models for Program Testing},
author = {Yifeng He and Jiabo Huang and Yuyang Rong and Yiwen Guo and Ethan Wang and Hao Chen},
booktitle = {ISSTA},
year = {2024},
doi = {10.1145/3650212.3680342}
}