一句话：提出 FailMapper，用九类失败场景和 MCTS 引导单元测试生成，显著提升缺陷触发能力。
现有单元测试生成多追求高代码覆盖率，但高覆盖未必能有效发现缺陷；而按缺陷属性引导又受缺陷类型多样限制。本文观察到失败作为缺陷结果仅呈现有限场景，提出 FailMapper：总结九类失败场景并设计对应失败触发测试策略，并用蒙特卡洛树搜索（MCTS）探索可能导致失败的故障，以生成更易触发缺陷的测试。实验在 Defects4J 的 50 个已知缺陷上检出 40 个，远超 EvoSuite、Randoop、CoverUp、HITS、SymPrompt 的最多 12 个；在 GitBug-Java 与 Bears 的 20 个缺陷中检出 12 个，并从两个 Apache 项目报告 36 个潜在问题，其中 14 个已确认。
The automation of unit test generation has become a critical task for improving the overall efficiency of software development and testing. Many existing techniques attempt to generate a sufficient number of test cases to achieve high code coverage. However, it has been shown that a high coverage does not necessarily guarantee effective bug discovery. A potential enhancement is to guide the unit test generation based on bug properties. However, this solution is challenged by the large number and diversity of bug types, making it difficult to comprehensively summarize bug properties.We observe that failures, presented as the results of bugs, manifest in a limited number of scenarios. Therefore, instead of bug properties, in this paper, we propose an innovative framework, named FailMapper, which uses failure scenarios to guide the generation of unit tests. We summarize nine failure scenarios and design the corresponding failure-triggering test strategies. This significantly improves the efficacy of generating test cases towards triggering bugs. To systematically explore possible failure scenarios, FailMapper employs the Monte Carlo Tree Search algorithm to search for the faults that may lead to a failure. Experiments demonstrate that, on 50 known bugs in the Defects4J benchmark, FailMapper can detect many more bugs than five typical unit testing approaches, including EvoSuite, Randoop, CoverUp, HITS, and SymPrompt (40 versus at most 12, out of all 50 bugs). Meanwhile, FailMapper detects 12 out of 20 bugs in the GitBug-Java and Bears-benchmark datasets. We reveal 36 potential issues from 2 Apache projects, and 14 of them have been confirmed as bugs, further demonstrating FailMapper’s effectiveness. The experimental results show that our new framework can significantly enhance the overall efficacy of unit testing.
DOI 原文 ·
@inproceedings{paperbot9452,
title = {FailMapper: Automated Generation of Unit Tests Guided by Failure Scenarios},
author = {Ruiqi Dong and Zehang Deng and Xiaogang Zhu and Xiaoning Du and Huai Liu and Shaohua Wang and Sheng Wen and Yang Xiang},
booktitle = {ASE},
year = {2025},
doi = {10.1109/ase63991.2025.00197}
}