一句话：实证评估五款 Java 静态缺陷检测器对真实空指针异常的有效性：102 个缺陷仅检出 30 个（29.4%）。
空指针异常（NPE）广泛存在于各编程领域。本文以经验研究方式评估五款 Java 静态缺陷检测器（CheckerFramework、ERADICATE、INFER、NULLAWAY、SPOTBUGS）在真实项目中发现 NPE 的有效性。作者从 BUGSWARM 与 DEFECTS4J 数据集中选取 42 个开源项目的 102 个可复现 NPE，采用两种已有判定方法，并提出两种利用堆栈轨迹与代码覆盖率的新判定方法，同时分类工具能力与缺陷特征。结果显示，所有工具合计仅发现 30 个缺陷（29.4%），其中大多数由 ERADICATE 发现。基于观察，作者讨论了提升工具有效性与实用性的机会。
Static bug detectors aim at helping developers to automatically find and prevent bugs. In this experience paper, we study the effectiveness of static bug detectors at identifying Null Pointer Dereferences or Null Pointer Exceptions (NPEs). NPEs pervade all programming domains from systems to web development. Specifically, our study measures the effectiveness of five Java static bug detectors: CheckerFramework, ERADICATE, INFER, NULLAWAY, and SPOTBUGS. We conduct our study on 102 real-world and reproducible NPEs from 42 open-source projects found in the BUGSWARM and DEFECTS4J datasets. We apply two known methods to determine whether a bug is found by a given tool, and introduce two new methods that leverage stack trace and code coverage information. Additionally, we provide a categorization of the tool’s capabilities and the bug characteristics to better understand the strengths and weaknesses of the tools. Overall, the tools under study only find 30 out of 102 bugs (29.4%), with the majority found by ERADICATE. Based on our observations, we identify and discuss opportunities to make the tools more effective and useful.
DOI 原文 ·
@inproceedings{paperbot6637,
title = {On the Real-World Effectiveness of Static Bug Detectors at Finding Null Pointer Exceptions},
author = {David A. Tomassi and Cindy Rubio-Gonzalez},
booktitle = {ASE},
year = {2021},
doi = {10.1109/ase51524.2021.9678535}
}