paperbot · PL 论文追踪

全部论文 待读 收藏

P2 Synthesizing File-Level Data for Unit Test Generation with Chain-of-Thoughts via Self-Debugging

通过自调试的思维链合成文件级数据用于单元测试生成
ASE Proceedings of the 41st IEEE/ACM International Conference on Automated Software Engineering2026匹配度 2/10 · 该论文聚焦于 LLM 单元测试生成、自调试数据蒸馏与模型微调,属于软件测试/代码生成方向,而非用户核心关注的“静态分析用于 ML/Agentic 程序”或“LLM/Agent 用于静态分析与形式化验证”。它也不涉及浮点程序分析或贝叶斯程序分析,因此与研究方向匹配度很低。引用 0 inbox
Ziyue Hua, Tianyu Chen, Yeyun Gong, Shuai Lu, Peng Cheng, Qinglin Zhu, Yibo He, Yingjie Fu, Wenpin Jiao, Wei Yang, Tao Xie
crossref: 2026-10-09

AI 速览

一句话:提出自调试数据蒸馏方法,生成带忠实思维链的高质量单元测试训练数据,并微调模型显著提升测试生成效果。

单元测试自动生成对软件质量保障至关重要,但现有符号执行、搜索和LLM方法难以产出带正确断言与可靠思维链(CoT)的测试。本文指出训练数据缺口:仓库挖掘测试缺少开发者CoT,LLM蒸馏CoT常不准确或不完整。为此提出自调试数据蒸馏方法:结合引导式测试修复(迭代修复测试)与CoT压缩(精简CoT为简洁解释)。在开源项目上构建74,518个高质量样本数据集,用于基座模型有监督微调。评估显示微调模型测试断言通过率36.17%、分支覆盖率43.90%、变异得分88.66%,显著优于o4-mini等模型。

问题
单元测试自动生成中,现有方法难以产出具有正确有意义断言和可靠思维链(CoT)解释的高质量测试;训练数据也存在缺口:仓库挖掘测试缺开发者CoT,LLM蒸馏CoT常不准确或不完整。
方法
提出自调试数据蒸馏流水线,结合引导式测试修复(按错误、失败、覆盖率步骤的启发式循环让模型诊断并迭代修复测试)与CoT压缩,将原始和调试CoT精简为直接论证正确测试的简洁解释,并用于构建大规模UT训练数据集。
评估
在大量开源项目上构建74,518个高质量样本,对基座模型进行有监督微调;评估显示微调模型在测试断言通过率、分支覆盖率和变异得分上显著优于o4-mini等先进商业模型。
原文摘要(Abstract)

Automatic unit test (UT) generation is essential for software quality assurance, but existing approaches--including symbolic execution, search-based approaches, and recent LLM-based generators--struggle to produce human-quality tests with correct, meaningful assertions and reliable chain-of-thought (CoT) explanations. We identify a gap in UT training data: repository-mined tests lack developer CoTs, while LLM-distilled CoTs are often incorrect or incomplete. To address this issue, we propose a novel data-distillation approach that uses self-debugging to produce high-quality UT training examples paired with faithful CoTs. Our approach combines (1) guided test repair, a heuristic loop (error-, failure-, and coverage-focused steps) that asks the used model to diagnose and iteratively fix generated tests, and (2) CoT compression, which compacts original and debugging CoTs into concise explanations that directly justify correct tests. We apply this pipeline to a large corpus of open-source projects to construct a dataset of 74,518 high-quality examples, and then use it for supervised fine-tuning of a base model. An empirical evaluation shows that the fine-tuned model achieves high UT generation effectiveness: it attains a pass rate of 36.17% on test assertions, a branch coverage of 43.90%, and a mutation score of 88.66%, substantially higher than state-of-the-art commercial models like o4-mini.

链接与引用

DOI 原文 · PDF(开放获取) ·

BibTeX
@inproceedings{paperbot12423,
  title = {Synthesizing File-Level Data for Unit Test Generation with Chain-of-Thoughts via Self-Debugging},
  author = {Ziyue Hua and Tianyu Chen and Yeyun Gong and Shuai Lu and Peng Cheng and Qinglin Zhu and Yibo He and Yingjie Fu and Wenpin Jiao and Wei Yang and Tao Xie},
  booktitle = {ASE},
  year = {2026},
  doi = {10.1145/3832783.3837477}
}