一句话：系统刻画基于语义的自动程序修复中的过拟合现象，发现其与启发式修复类似地以多种形式出现。
自动程序修复（APR）可分为基于语义与基于启发式两类。基于语义的 APR 利用符号执行和测试套件提取语义约束，再通过程序合成产生满足约束的补丁；二者都依赖“通过全部测试即正确修复”的假设，因此可能产生过拟合补丁——它们能通过现有测试，却无法泛化。本文聚焦基于语义的 APR，使用适合评估修复质量的 IntroClass 与 Codeflaws 基准，对过拟合问题进行系统性的实证刻画与分析。研究发现，与基于启发式的 APR 类似，基于语义的 APR 中过拟合也会以多种不同方式出现，说明补丁正确性问题在该类技术中同样严峻，需要更严谨的评估方法。
Existing APR techniques can be generally divided into two families: semantics- vs. heuristics-based. Semantics-based APR uses symbolic execution and test suites to extract semantic constraints, and uses program synthesis to synthesize repairs that satisfy the extracted constraints. Heuristic-based APR generates large populations of repair candidates via source manipulation, and searches for the best among them. Both families largely rely on a primary assumption that a program is correctly patched if the generated patch leads the program to pass all provided test cases. Patch correctness is thus an especially pressing concern. A repair technique may generate overfitting patches, which lead a program to pass all existing test cases, but fails to generalize beyond them. In this work, we revisit the overfitting problem with a focus on semantics-based APR techniques, complementing previous studies of the overfitting problem in heuristics-based APR. We perform our study using IntroClass and Codeflaws benchmarks, two datasets well-suited for assessing repair quality, to systematically characterize and understand the nature of overfitting in semantics-based APR. We find that similar to heuristics-based APR, overfitting also occurs in semantics-based APR in various different ways.
DOI 原文 ·
@inproceedings{paperbot4490,
title = {Overfitting in semantics-based automated program repair},
author = {Xuan-Bach D. Le and Ferdian Thung and David Lo and Claire Le Goues},
booktitle = {ICSE},
year = {2018},
doi = {10.1145/3180155.3182536}
}