一句话：提出统一模型 CORE，融合覆盖率特征、专家特征与测试/焦点方法的语义表示，显著提升巧合正确性测试用例的检测效果。
巧合正确性（coincidental correctness, CC）指测试用例虽然通过却掩盖了隐藏故障，容易误导开发者。已有工作主要分别依赖覆盖率特征或专家特征进行 CC 检测，但尚未充分探索两类特征的结合潜力，也未充分利用测试代码与焦点方法的丰富语义信息。为此，作者提出统一模型 CORE，将覆盖率特征、专家特征与测试方法和焦点方法的语义表示集成在一起，以更准确地识别 CC 测试用例。作者在广泛使用的 Defects4J 基准上，与六个当前最优基线进行了全面对比。实验结果表明，CORE 在 CC 检测准确率上优于所有基线，F1 分数平均提升约 40%。消融实验进一步说明覆盖率、专家特征与语义信息三者均有贡献。此外，CORE 还能提升基于频谱与基于变异的故障定位效果，例如在重标注策略下 Dstar 公式提升约 50%，MUSE 方法提升约 44%。
Coincidental correctness (CC) can be misleading for developers because it gives the impression that the code is functioning correctly when there are hidden faults. To mitigate the negative impacts of CC test cases, extensive research has been conducted on their detection, employing either coverage-based or expert-based features. These studies have yielded promising results. Coverage and expert features each provide unique insights into program execution, yet the literature has not fully explored the combined potential of these two feature sets to enhance the detection of CC. Additionally, the rich semantics of the test code and focal method have not been fully utilized. Therefore, we propose to build a unified model, CORE, that integrates coverage and expert features with semantic representations of test and focal methods to improve the detection of CC test cases. We make a comprehensive evaluation with six state-of-the-art baselines on the widely-used Defects4J benchmark. The experimental results show that CORE outperforms the baselines in terms of CC detection accuracy, with a substantial improvement (i.e., 40% improvement on average in terms of F1 score). Then, we conduct the ablation experiment to show that the coverage, expert, and semantics contribute to CORE. CORE can also improve the effectiveness of spectrum-based and mutation-based fault localization performance (e.g., 50% improvements for spectrum-based formula Dstar and 44% improvements for mutation-based method MUSE under relabeling strategy).
DOI 原文 ·
@inproceedings{paperbot8401,
title = {Combining Coverage and Expert Features with Semantic Representation for Coincidental Correctness Detection},
author = {Huan Xie and Yan Lei and Maojin Li and Meng Yan and Sheng Zhang},
booktitle = {ASE},
year = {2024},
doi = {10.1145/3691620.3695542}
}