一句话：大规模实证揭示测试套件度量（语句覆盖、分支覆盖、变异得分）与自动修复可靠性的相关性，语句覆盖趋势最强。
自动程序修复有望降低调试成本，但生成补丁的质量（尤其是是否引入回归）仍是关键问题。本文提出并回答：软件测试中传统的测试套件度量能否用于自动程序修复？作者在真实世界软件上开展了迄今规模最大的实验，首次系统研究测试套件度量（语句覆盖、分支覆盖与变异得分）与修复可靠性之间的相关性，其中修复可靠性用补丁引发回归的可能性衡量。结果显示，随着测试套件度量的提高，修复的可靠性总体呈上升趋势，且该趋势在语句覆盖上最为显著。这表明软件测试中的传统测试套件度量也可用于提升程序修复的可靠性，为通过控制测试套件质量来筛选或改善补丁提供了实证依据。
Automated program repair has attracted attention due to its potential to reduce debugging cost. Prior works show the feasibility of automated repair, and the research focus is gradually shifting towards the quality of generated patches. One promising direction is to control the quality of generated patches by controlling the quality of test-suites used. In this paper, 1we investigate the question: "Can traditional test-suite metrics used in software testing be used for automated program repair?". We empirically investigate the effectiveness of test-suite metrics (statement / branch coverage and mutation score) in controlling the reliability of repairs (the likelihood that repairs cause regressions). We conduct the largest-scale experiments to date with real-world software, and perform the first correlation study between test-suite metrics and the reliability of generated repairs. Our results show that by increasing test-suite metrics, the reliability of repairs tend to increase. Particularly, such trend is most strongly observed in statement coverage. This implies that traditional test-suite metrics used in software testing can also be used to improve the reliability of repairs in program repair.
DOI 原文 ·
@inproceedings{paperbot4476,
title = {A correlation study between automated program repair and test-suite metrics},
author = {Jooyong Yi and Shin Hwei Tan and Sergey Mechtaev and Marcel Böhme and Abhik Roychoudhury},
booktitle = {ICSE},
year = {2018},
doi = {10.1145/3180155.3182517}
}