一句话：提出以存活变异体为引导的全自动 LLM 测试套件增强方法，显著提升变异得分并减少代码生成评估的假阳性。
现有基于大语言模型的单元测试生成研究多以代码覆盖率作为提升测试套件有效性的主要指标，但覆盖率可达约 80% 时，与缺陷检测能力更相关的变异得分却仅约 35%。MuTAP 和 μTestGen 等已有工作尝试利用存活变异体提升有效性，但其评估仅限依赖内置函数和标准库的简单独立方法；对于依赖其他类、涉及复杂用户自定义类型的非独立方法，其有限的上下文信息与基础修复机制能否泛化尚不明确，且其依赖语言特定的基于规则的变异技术，跨语言适配需要额外配置与成本。为解决该问题，本文提出一种全自动的 LLM 方法：将变异测试与测试用例生成相结合，以焦点方法信息为输入，针对初始测试套件运行后识别出的存活变异体生成测试用例，并采用多种提示技术、丰富上下文信息与高级修复机制以支持非独立方法。评估覆盖 1035 个焦点方法（分为独立与非独立两类），平均变异得分分别提升 16.04% 和 8.11%。作者进一步在基于 LLM 的代码生成中验证实际影响：增强测试套件后，独立与非独立方法的 pass@1 平均分别下降 0.3152 和 0.1772，表明该方法能有效减少因测试用例不足造成的代码生成评估假阳性。
Large Language Models (LLMs) have gained significant traction in software engineering for automating tasks such as unit test generation. Most existing studies prioritize code coverage as the primary metric for enhancing test suite effectiveness. However, prior research has shown that although code coverage can reach approximately 80%, the mutation score, which generally exhibits a stronger correlation with defect detection effectiveness, attains only about 35%. This gap highlights the need to enhance test suite effectiveness guided by mutation score rather than code coverage. Recent studies, including MuTAP and Mu tGe n, explored the use of survived mutants to enhance test suite effectiveness. However, their evaluations were limited to simple standalone methods that rely on built-in functions and standard libraries. Non-standalone methods, which depend on other classes and involve complex user-defined types, are more intricate and commonly found in real-world projects. The limited contextual information and basic repair mechanisms in their prompt designs make it unclear whether their performance can generalize to non-standalone methods. Moreover, the two studies rely on existing language-specific, rule-based mutation techniques, which require specific configurations and incur additional costs when adapting to other programming languages. To bridge this gap, we propose a novel, fully automatic LLM-based approach to enhance test suite effec-tiveness, guided by survived mutants. The approach augments initial test suites by integrating mutation testing with test case generation. It takes focal method information as input and generates test cases targeting survived mutants identified from applying the initial test suites. Our approach incorporates multiple prompt techniques, rich contextual information, and an advanced repair mechanism to effectively generate test cases for non-standalone methods. The evaluation covers 1,035 focal methods, categorized as standalone or non-standalone. On average, the mutation score increases by 16.04% for standalone methods and 8.11% for non-standalone methods. We validate the practical impact of augmented test suites in LLM-based code generation. After test suite augmentation, pass@1 decreased by 0.3152 and 0.1772 on average for standalone and non-standalone methods, respectively, indicating the effectiveness of our approach in reducing false positives caused by insufficient test cases in code generation evaluation.
@article{paperbot3777,
title = {Beyond Coverage: Automatic Test Suite Augmentation for Enhanced Effectiveness using Large Language Models},
author = {Zeyu Lu and Peng Zhang and Yuge Nie and Yibiao Yang and Yutian Tang and Chun Yong Chong and Yuming Zhou},
journal = {Proceedings of the ACM on Programming Languages},
volume = {10},
number = {OOPSLA1},
year = {2026},
doi = {10.1145/3798251}
}