paperbot · PL 论文追踪

全部论文 待读 收藏

P2 FDG: a precise measurement of fault diagnosability gain of test cases

FDG:测试用例故障可诊断性增益的精确度量
ISSTA Proceedings of the 31st ACM SIGSOFT International Symposium on Software Testing and Analysis2022匹配度 2/10 · 该论文属于软件测试与故障定位方向,关注测试用例对故障定位的可诊断性增益,与用户画像核心的静态分析、LLM/Agent 结合、浮点程序分析及贝叶斯程序分析均无直接关联;仅在软件工程/程序调试工具层面存在弱相关。引用 6 inbox
Gabin An, Shin Yoo
crossref: 2026-09-12

AI 速览

一句话:提出 FDG 指标精确度量单个测试用例对故障定位的可诊断性增益,并用于测试套件增强。

故障定位(FL)效果高度依赖测试套件质量,因此需精确衡量每个测试用例加入后带来的诊断能力。现有面向 FL 的测试用例优先级指标未充分利用 FL 中间结果。本文提出 FDG,度量单个测试用例的故障可诊断性增益,并利用正在进行的 FL 结果,将注意力导向程序中仍缺乏诊断信息的部分。在 Defects4J 上评估表明,当仅有少量失败测试用例(平均 2.3 个)时,FDG 能优先选择 EvoSuite 自动生成的用例来增强薄弱套件;仅需十次人工判断断言正确性,即可使 FL 的 acc@1 和 acc@10 平均提升 11.6 倍与 2.2 倍。

问题
如何精确度量单个测试用例对故障定位的诊断能力,以支持测试用例优先级、选择和薄弱测试套件的增强。
方法
提出 FDG 指标,基于对现有 FL 优先级指标的分析,并利用正在进行的故障定位结果来强调仍缺乏诊断信息的程序部分;在 Defects4J 上用 EvoSuite 生成用例进行测试套件增强评估。
评估
在 Defects4J 上,仅给少量失败用例并需十次人工断言判定,FDG 增强使 acc@1 和 acc@10 平均提升 11.6 倍与 2.2 倍。
原文摘要(Abstract)

The performance of many Fault Localisation (FL) techniques directly depends on the quality of the used test suites. Consequently, it is extremely useful to be able to precisely measure how much diagnostic power each test case can introduce when added to a test suite used for FL. Such a measure can help us not only to prioritise and select test cases to be used for FL, but also to effectively augment test suites that are too weak to be used with FL techniques. We propose FDG, a new measure of Fault Diagnosability Gain for individual test cases. The design of FDG is based on our analysis of existing metrics that are designed to prioritise test cases for better FL. Unlike other metrics, FDG exploits the ongoing FL results to emphasise the parts of the program for which more information is needed. Our evaluation of FDG with Defects4J shows that it can successfully help the augmentation of test suites for better FL. When given only a few failing test cases (2.3 test cases on average), FDG can effectively augment the given test suite by prioritising the test cases generated automatically by EvoSuite: the augmentation can improve the acc@1 and acc@10 of the FL results by 11.6x and 2.2x on average, after requiring only ten human judgements on the correctness of the assertions EvoSuite generates.

链接与引用

DOI 原文 · PDF(开放获取) ·

BibTeX
@inproceedings{paperbot7437,
  title = {FDG: a precise measurement of fault diagnosability gain of test cases},
  author = {Gabin An and Shin Yoo},
  booktitle = {ISSTA},
  year = {2022},
  doi = {10.1145/3533767.3534370}
}