paperbot · PL 论文追踪

全部论文 待读 收藏

P2 Clarifying Semantics of In-Context Examples for Unit Test Generation

澄清用于单元测试生成的上下文示例的语义
ASE 2025 40th IEEE/ACM International Conference on Automated Software Engineering (ASE)2025匹配度 4/10 · 论文结合程序分析与 LLM 重写来精炼单元测试示例,属于 PL+AI/LLM 辅助软件工程,与用户关注的 LLM/Agent 改进静态分析或形式化验证仅有边缘关联;其核心任务是单元测试生成与 ICL 示例优化,不涉及 Agentic 程序、ML 系统静态分析、浮点程序分析或贝叶斯程序分析。引用 3 inbox
Chen Yang, Lin Yang, Ziqi Wang, Dong Wang, Jianyi Zhou, Junjie Chen
crossref: 2026-09-12

AI 速览

一句话:提出 CLAST,结合程序分析与 LLM 重写精炼单元测试,提升其作为上下文示例的语义清晰度与 ICL 生成效果。

LLM 通过上下文学习(ICL)生成单元测试时,上下文示例质量会显著影响效果,但示例常结构不佳或语义不清。本文提出 CLAST,将复杂测试分解为逻辑更清晰的测试,并结合程序分析与 LLM 重写提升语义清晰度,以增强其作为上下文示例的效用。在四个开源和三个工业项目上的评估表明,CLAST 在保持测试有效性的同时,语义清晰度优于 UTgen;UTgen 会降低编译成功率、通过率、覆盖率和变异得分。超过 85.33% 的用户研究参与者更偏好 CLAST 精炼后的测试。将 CLAST 测试作为示例还能提升 RAGGen、TELPA 等 ICL 生成方法。

问题
LLM 基于上下文学习生成单元测试时,上下文示例的结构和语义清晰度不足会限制生成测试的质量。
方法
CLAST 将复杂测试分解为逻辑更清晰的测试,并结合程序分析与 LLM 重写来系统精炼单元测试,提升其语义清晰度。
评估
在四个开源和三个工业项目上,与 UTgen 对比测试有效性、语义清晰度及用户偏好,并验证精炼测试对 RAGGen、TELPA 等 ICL 方法的增益。
原文摘要(Abstract)

Recent advances in large language models (LLMs) have enabled promising performance in unit test generation through in-context learning (ICL). However, the quality of in-context examples significantly influences the effectiveness of generated tests—poorly structured or semantically unclear test examples often lead to suboptimal outputs. In this paper, we propose CLAST, a novel technique that systematically refines unit tests to improve their semantic clarity, thereby enhancing their utility as in-context examples. The approach decomposes complex tests into logically clearer ones and improves semantic clarity through a combination of program analysis and LLM-based rewriting. We evaluated CLAST on four open-source and three industrial projects. The results demonstrate that CLAST largely outperforms UTgen, the state-of-the-art refinement technique, in both preserving test effectiveness and enhancing semantic clarity. Specifically, CLAST fully retains the original effectiveness of unit tests, while UTgen reduces compilation success rate (CSR), pass rate (PR), test coverage (Cov), and mutation score (MS) by an average of 12.90%, 35.82%, 4.65%, and 5.07%, respectively. Over 85.33% of participants in our user study preferred the semantic clarity of CLAST-refined tests. Notably, incorporating CLAST-refined tests as examples effectively improves ICL-based unit test generation approaches such as RAGGen and TELPA, resulting in an average increase of 25.97% in CSR, 28.22% in PR, and 45.99% in Cov for generated tests, compared to incorporating UTgen-refined tests. The insights from the follow-up user study not only reinforce CLAST’s potential impact in software testing practice but also illuminate avenues for future research.

链接与引用

DOI 原文 ·

BibTeX
@inproceedings{paperbot9426,
  title = {Clarifying Semantics of In-Context Examples for Unit Test Generation},
  author = {Chen Yang and Lin Yang and Ziqi Wang and Dong Wang and Jianyi Zhou and Junjie Chen},
  booktitle = {ASE},
  year = {2025},
  doi = {10.1109/ase63991.2025.00250}
}