paperbot · PL 论文追踪

全部论文 待读 收藏

P2 Knowledge Matters: Injecting Project and Testing Knowledge into LLM-based Unit Test Generation

知识至关重要:将项目知识与测试知识注入基于 LLM 的单元测试生成
ICSE Proceedings of the 2026 IEEE/ACM 48th International Conference on Software Engineering2026匹配度 4/10 · 论文将静态分析用于增强 LLM 单元测试生成,属于 LLM 辅助软件工程,但与用户核心关注的“ML/Agentic 程序的静态分析”“LLM 改进静态分析/形式化验证”“浮点或贝叶斯程序分析”均不直接对应。其贡献更多在测试生成与提示工程,而非程序分析理论或验证工具。引用 0 inbox
Anji Li, Mingwei Liu, Zhenxi Chen, Zheng Pei, Zike Li, Dekun Dai, Yanlin Wang, Zibin Zheng
crossref: 2026-09-12

AI 速览

一句话:提出 KTester,融合项目结构与测试领域知识,显著提升 LLM 单元测试的正确性与可维护性。

LLM 自动生成单元测试前景可观,但在真实项目中常难以兼顾正确性与可维护性,原因在于模型缺少项目上下文与测试设计知识。本文提出 KTester:先用静态分析抽取项目结构和使用知识,为模型提供丰富上下文;再以测试领域知识引导,将测试用例设计与测试方法生成分离,并结合多视角提示策略促使 LLM 考虑多样测试启发式,最终按结构化模板输出测试以提升清晰度与可维护性。作者在多个开源项目上,用自动正确性/覆盖率指标与人工可读性、可维护性评审对比最强 LLM 基线。结果表明 KTester 在六项指标上显著领先,执行通过率提升 5.69%、行覆盖率提升 8.83%,且耗时更少、用例更少,人工评分在正确性、可读性与可维护性上亦显著更高。

问题
基于 LLM 的单元测试生成在真实项目中往往难以同时保证测试的正确性与可维护性,因为模型缺乏项目特定上下文和测试设计知识。
方法
通过静态分析提取项目结构和使用知识作为上下文,并以测试领域知识将测试用例设计与测试方法生成分离,结合多视角提示策略和结构化模板引导 LLM 生成测试。
评估
在多个开源项目上与当前最强的 LLM 基线对比,使用自动正确性与覆盖率指标,并开展人工研究评估可读性与可维护性;在六项指标上均取得显著提升。
原文摘要(Abstract)

Automated unit test generation using large language models (LLMs) holds great promise but often struggles with generating tests that are both correct and maintainable in real-world projects. This paper presents KTester, a novel framework that integrates project-specific knowledge and testing domain knowledge to enhance LLM-based test generation. Our approach first extracts project structure and usage knowledge through static analysis, which provides rich context for the model. It then employs a testing-domain-knowledge-guided separation of test case design and test method generation, combined with a multi-perspective prompting strategy that guides the LLM to consider diverse testing heuristics. The generated tests follow structured templates, improving clarity and maintainability. We evaluate KTester on multiple open-source projects, comparing it against state-of-the-art LLM-based baselines using automatic correctness and coverage metrics, as well as a human study assessing readability and maintainability. Results demonstrate that KTester significantly outperforms existing methods across six key metrics, improving execution pass rate by 5.69% and line coverage by 8.83% over the strongest baseline, while requiring less time and generating fewer test cases. Human evaluators also rate the tests produced by KTester significantly higher in terms of correctness, readability, and maintainability, confirming the practical advantages of our knowledge-driven framework.

链接与引用

DOI 原文 · PDF(开放获取) ·

BibTeX
@inproceedings{paperbot11886,
  title = {Knowledge Matters: Injecting Project and Testing Knowledge into LLM-based Unit Test Generation},
  author = {Anji Li and Mingwei Liu and Zhenxi Chen and Zheng Pei and Zike Li and Dekun Dai and Yanlin Wang and Zibin Zheng},
  booktitle = {ICSE},
  year = {2026},
  doi = {10.1145/3744916.3787769}
}