一句话:提出静动态分析引导的单元测试生成策略,显著提升 Java 空指针异常的发现效果。
本文是一篇经验论文,研究如何增强自动单元测试生成以更有效发现 Java 空指针异常(NPE)。作者观察到,Randoop、EvoSuite 等现有工具以高代码覆盖为主要目标,未必能触发多样的 NPE。为此,论文提出一种 NPE 引导策略,结合静态分析与动态分析,引导测试用例生成器聚焦于更可能触发 NPE 的场景,并在 EvoSuite 之上实现了工具 NpeTest。评估使用从 96 个真实项目收集的 108 个 NPE 基准,将 EvoSuite 的 NPE 复现率从 56.9% 提升到 78.9%,相对提升 38.7%;此外,NpeTest 还在一个工业项目中检测出 89 个此前未知的 NPE。
In this experience paper, we share our experience on enhancing automatic unit test generation to more effectively find Java null pointer exceptions (NPEs). NPEs are among the most common and critical errors in Java applications. However, as we demonstrate in this paper, existing unit test generation tools such as Randoop and EvoSuite are not sufficiently effective at catching NPEs. Specifically, their primary strategy of achieving high code coverage does not necessarily result in triggering diverse NPEs in practice. In this paper, we detail our observation on the limitations of current state-of-the-art unit testing tools in terms of NPE detection and introduce a new strategy to improve their effectiveness. Our strategy utilizes both static and dynamic analyses to guide the test case generator to focus specifically on scenarios that are likely to trigger NPEs. We implemented this strategy on top of EvoSuite, and evaluated our tool, NpeTest, on 108 NPE benchmarks collected from 96 real-world projects. The results show that our NPE-guidance strategy can increase EvoSuite's reproduction rate of the NPEs from 56.9% to 78.9%, a 38.7% improvement. Furthermore, NpeTest successfully detected 89 previously unknown NPEs from an industry project.