一句话:通过调研100余名开发者并定量评估现有测试套件修复工具,揭示增强自动补丁信任的需求与差距。
自动程序修复利用学习、搜索与语义分析自动修复缺陷,但对自动生成补丁的信任不足阻碍其广泛应用。本文通过问卷调研100余名软件从业者,了解增强补丁信任所需的工件与流程设置;并依据开发者偏好,对现有基于测试套件的修复工具进行定量评估。结果显示,这些工具无法在top-10排名与1小时的可接受时限内产出高质量补丁。作者结合定性调研与定量实验指出:在可接受时间界内产生修复高度依赖对足够丰富搜索空间的抽象表示;开发者虽认可额外输入的价值,但对显著的人在回路交互兴趣不高。这些发现为程序修复研究提供了可操作的改进方向。
Automated program repair is an emerging technology that seeks to automatically rectify bugs and vulnerabilities using learning, search, and semantic analysis. Trust in automatically generated patches is necessary for achieving greater adoption of program repair. Towards this goal, we survey more than 100 software practitioners to understand the artifacts and setups needed to enhance trust in automatically generated patches. Based on the feedback from the survey on developer preferences, we quantitatively evaluate existing test-suite based program repair tools. We find that they cannot produce high-quality patches within a top-10 ranking and an acceptable time period of 1 hour. The developer feedback from our qualitative study and the observations from our quantitative examination of existing repair tools point to actionable insights to drive program repair research. Specifically, we note that producing repairs within an acceptable time-bound is very much dependent on leveraging an abstract search space representation of a rich enough search space. Moreover, while additional developer inputs are valuable for generating or ranking patches, developers do not seem to be interested in a significant human-in-the-loop interaction.