一句话:提出 FDG 指标精确度量单个测试用例对故障定位的可诊断性增益,并用于测试套件增强。
故障定位(FL)效果高度依赖测试套件质量,因此需精确衡量每个测试用例加入后带来的诊断能力。现有面向 FL 的测试用例优先级指标未充分利用 FL 中间结果。本文提出 FDG,度量单个测试用例的故障可诊断性增益,并利用正在进行的 FL 结果,将注意力导向程序中仍缺乏诊断信息的部分。在 Defects4J 上评估表明,当仅有少量失败测试用例(平均 2.3 个)时,FDG 能优先选择 EvoSuite 自动生成的用例来增强薄弱套件;仅需十次人工判断断言正确性,即可使 FL 的 acc@1 和 acc@10 平均提升 11.6 倍与 2.2 倍。
The performance of many Fault Localisation (FL) techniques directly depends on the quality of the used test suites. Consequently, it is extremely useful to be able to precisely measure how much diagnostic power each test case can introduce when added to a test suite used for FL. Such a measure can help us not only to prioritise and select test cases to be used for FL, but also to effectively augment test suites that are too weak to be used with FL techniques. We propose FDG, a new measure of Fault Diagnosability Gain for individual test cases. The design of FDG is based on our analysis of existing metrics that are designed to prioritise test cases for better FL. Unlike other metrics, FDG exploits the ongoing FL results to emphasise the parts of the program for which more information is needed. Our evaluation of FDG with Defects4J shows that it can successfully help the augmentation of test suites for better FL. When given only a few failing test cases (2.3 test cases on average), FDG can effectively augment the given test suite by prioritising the test cases generated automatically by EvoSuite: the augmentation can improve the acc@1 and acc@10 of the FL results by 11.6x and 2.2x on average, after requiring only ten human judgements on the correctness of the assertions EvoSuite generates.