一句话:提出 CLAST,结合程序分析与 LLM 重写精炼单元测试,提升其作为上下文示例的语义清晰度与 ICL 生成效果。
LLM 通过上下文学习(ICL)生成单元测试时,上下文示例质量会显著影响效果,但示例常结构不佳或语义不清。本文提出 CLAST,将复杂测试分解为逻辑更清晰的测试,并结合程序分析与 LLM 重写提升语义清晰度,以增强其作为上下文示例的效用。在四个开源和三个工业项目上的评估表明,CLAST 在保持测试有效性的同时,语义清晰度优于 UTgen;UTgen 会降低编译成功率、通过率、覆盖率和变异得分。超过 85.33% 的用户研究参与者更偏好 CLAST 精炼后的测试。将 CLAST 测试作为示例还能提升 RAGGen、TELPA 等 ICL 生成方法。
Recent advances in large language models (LLMs) have enabled promising performance in unit test generation through in-context learning (ICL). However, the quality of in-context examples significantly influences the effectiveness of generated tests—poorly structured or semantically unclear test examples often lead to suboptimal outputs. In this paper, we propose CLAST, a novel technique that systematically refines unit tests to improve their semantic clarity, thereby enhancing their utility as in-context examples. The approach decomposes complex tests into logically clearer ones and improves semantic clarity through a combination of program analysis and LLM-based rewriting. We evaluated CLAST on four open-source and three industrial projects. The results demonstrate that CLAST largely outperforms UTgen, the state-of-the-art refinement technique, in both preserving test effectiveness and enhancing semantic clarity. Specifically, CLAST fully retains the original effectiveness of unit tests, while UTgen reduces compilation success rate (CSR), pass rate (PR), test coverage (Cov), and mutation score (MS) by an average of 12.90%, 35.82%, 4.65%, and 5.07%, respectively. Over 85.33% of participants in our user study preferred the semantic clarity of CLAST-refined tests. Notably, incorporating CLAST-refined tests as examples effectively improves ICL-based unit test generation approaches such as RAGGen and TELPA, resulting in an average increase of 25.97% in CSR, 28.22% in PR, and 45.99% in Cov for generated tests, compared to incorporating UTgen-refined tests. The insights from the follow-up user study not only reinforce CLAST’s potential impact in software testing practice but also illuminate avenues for future research.
DOI 原文 ·
@inproceedings{paperbot9426,
title = {Clarifying Semantics of In-Context Examples for Unit Test Generation},
author = {Chen Yang and Lin Yang and Ziqi Wang and Dong Wang and Jianyi Zhou and Junjie Chen},
booktitle = {ASE},
year = {2025},
doi = {10.1109/ase63991.2025.00250}
}