一句话:提出因果测试,用反事实因果推断定位缺陷根因,并实现 Holmes 插件。
论文针对缺陷根因难以定位的问题,提出 Causal Testing(因果测试):它基于反事实因果理论,从程序执行中识别可能包含关键因果信息的执行集合,从而帮助开发者理解并修复错误行为。作者在 Defects4J 基准上评估,发现该方法可应用于 71% 的真实缺陷;在其中 77% 的缺陷上,它能帮助开发者找到根因。一项有 37 名开发者参与的对照实验表明,使用标准测试工具时参与者识别缺陷原因的成功率为 80%,使用因果测试后提升至 86%;参与者认为因果测试提供了 JUnit 等工具无法提供的有用信息。作者还实现了开源 Eclipse 插件 Holmes。
Understanding the root cause of a defect is critical to isolating and repairing buggy behavior. We present Causal Testing, a new method of root-cause analysis that relies on the theory of counterfactual causality to identify a set of executions that likely hold key causal information necessary to understand and repair buggy behavior. Using the Defects4J benchmark, we find that Causal Testing could be applied to 71% of real-world defects, and for 77% of those, it can help developers identify the root cause of the defect. A controlled experiment with 37 developers shows that Causal Testing improves participants' ability to identify the cause of the defect from 80% of the time with standard testing tools to 86% of the time with Causal Testing. The participants report that Causal Testing provides useful information they cannot get using tools such as JUnit. Holmes, our prototype, open-source Eclipse plugin implementation of Causal Testing, is available at http://holmes.cs.umass.edu/.