一句话：定义结构测试的覆盖缺口，实证其与故障检测负相关，并用依赖静态分析推荐测试焦点方法以缩小缺口。
结构代码覆盖率是常用的测试充分性指标，但其计算通常未考虑测试预言质量，可能高估测试套件的故障检测能力。本文将结构测试中的覆盖缺口形式化为被执行但未被任何测试预言观察到的程序结构比例。基于13个Java应用、1.6万测试用例和5.16万测试断言的实证研究表明，即使成熟测试套件，缺口也最高达51个百分点，平均34个百分点，且覆盖缺口与故障检测有效性呈强负相关。为缩小缺口，作者提出一种轻量级程序依赖静态分析，对可降低缺口的测试焦点方法进行排序推荐；以3.48万条断言为真实标注时，推荐器在前五名中覆盖了开发者所写焦点方法的三分之二。
Structural code coverage is a popular test adequacy metric that measures the percentage of program structure (e.g., statement, branch, decision) executed by a test suite. While structural coverage has several benefits, previous studies suggested that code coverage is not a good indicator of a test suite's fault-detection effectiveness as coverage computation does not consider test oracle quality. In this research, we formally define the coverage gap in structural testing as the percentage of program structure that is executed but not observed by any test oracles. Our large-scale empirical study of 13 Java applications, 16K test cases and 51.6K test assertions shows that even for mature test suites, the gap can be as high as 51 percentage points (pp) and 34pp on average. Our study reveals that the coverage gap strongly and negatively correlates with a test suite's fault-detection effectiveness. To mitigate gaps, we propose a lightweight static analysis of program dependencies to produce a ranked recommendation of test focus methods that can reduce the gap and improve test suite quality. When considering 34.8K assertions in the test suite as ground truth, the recommender suggests two-thirds of the focus methods written by developers within the top five recommendations.
@inproceedings{HossainDEN23,
title = {Measuring and Mitigating Gaps in Structural Testing},
author = {Soneya Binta Hossain and Matthew B. Dwyer and Sebastian Elbaum and Anh Nguyen-Tuong},
booktitle = {ICSE},
year = {2023},
doi = {10.1109/icse48619.2023.00147}
}