一句话：比较开发者提交的测试与用户提供的测试，评估二者对故障定位和自动程序修复技术评估的影响。
本文关注软件测试与调试技术的评估方法。论文指出，使用缺陷跟踪系统中的真实缺陷以及修复提交中的真实测试，虽然提高了真实性，但开发者修复后提交的测试可能包含比最初诊断缺陷时更多的信息，因而未必代表实践中开发者或工具可用的信息。为此，作者比较“开发者提供的测试”与“用户提供的测试”在故障定位和自动程序修复中的表现，分析不同测试来源如何影响评估结论。该研究强调，测试来源的选择会改变对技术有效性的判断，并影响故障定位与自动程序修复研究的实验设计。
To realistically evaluate a software testing or debugging technique, it must be run on defects and tests that are characteristic of those a developer would encounter in practice. For example, to determine the utility of a fault localization or automated program repair technique, it could be run on real defects from a bug tracking system, using real tests that are committed to the version control repository along with the fixes. Although such a methodology uses real tests, it may not use tests that are characteristic of the information a developer or tool would have in practice. The tests that a developer commits after fixing a defect may encode more information than was available to the developer when initially diagnosing the defect.
DOI 原文 ·
@inproceedings{paperbot4806,
title = {Comparing developer-provided to user-provided tests for fault localization and automated program repair},
author = {René Just and Chris Parnin and Ian Drosos and Michael D. Ernst},
booktitle = {ISSTA},
year = {2018},
doi = {10.1145/3213846.3213870}
}