一句话:实证评估 ErrorProne 与 SpotBugs 对 320 个真实 Java 缺陷的潜在与实际检出效果。
静态分析是发现软件缺陷的重要技术,但缺乏其在真实软件中有效性的证据。本文对流行的静态分析器 ErrorProne 与 SpotBugs 进行初步实证研究:从 BugSwarm 数据集选取 320 个真实 Java 缺陷,判断哪些缺陷可能被分析器发现,并统计实际检出数量。结果显示,ErrorProne 和 SpotBugs 分别可覆盖 30.3% 与 40.3% 的缺陷候选;两者较容易集成到使用 Maven 构建系统的多样项目中,但实际检出效果不佳,SpotBugs 仅成功发现一个缺陷。该研究揭示了静态分析工具在真实缺陷发现能力上的差距。
Static analysis is a powerful technique to find software bugs. In past years, a few static analysis tools have become available for developers to find certain kinds of bugs in their programs. However, there is no evidence on how effective the tools are in finding bugs in real-world software. In this paper, we present a preliminary study on the popular static analyzers ErrorProne and SpotBugs. Specifically, we consider 320 real Java bugs from the BugSwarm dataset, and determine which of these bugs can potentially be found by the analyzers, and how many are indeed detected. We find that 30.3% and 40.3% of the bugs are candidates for detection by ErrorProne and SpotBugs, respectively. Our evaluation shows that the analyzers are relatively easy to incorporate into the tool chain of diverse projects that use the Maven build system. However, the analyzers are not as effective detecting the bugs under study, with only one bug successfully detected by SpotBugs.