一句话:提出异构提示与执行反馈技术,自动为软件工程 issue 生成并选择复现测试。
许多软件工程 issue 缺少可运行的复现测试,阻碍问题定位与修复。本文研究如何自动生成此类测试。针对温度采样推理扩展多样性不足的问题,提出异构提示;针对 issue 解决前新代码尚不存在、难以利用执行反馈的问题,提出新的执行反馈途径。作者在 e-Otter++ 中实现这些技术,并在 TDD-Bench Verified 基准上评估,生成的测试平均 fail-to-pass 率达 63%,显著推进了该任务的最优水平。
A software engineering issue (SWE issue) is easier to resolve when accompanied by a reproduction test. Unfortunately, most issues do not come with functioning reproduction tests, so this paper explores how to generate them automatically. One technique that helps with this task is inference scaling, but traditional temperature-based scaling tends to lack diversity. This paper introduces heterogeneous prompting to address this problem. Another technique that helps with this task is execution feedback, but this is hampered by the fact that the new code (after issue resolution) to execute does not yet exist. This paper introduces novel approaches to get around that problem. We implemented our techniques in a new reproduction test generator called e-Otter++. Experiments show that e-Otter++ represents a leap ahead in the state-of-the-art for this problem, generating tests with an average fail-to-pass rate of 63% on the TDD-Bench Verified benchmark.
@inproceedings{paperbot11827,
title = {Heterogeneous Prompting and Execution Feedback for SWE Issue Test Generation and Selection},
author = {Toufique Ahmed and Jatin Ganhotra and Avraham Shinnar and Martin Hirzel},
booktitle = {ICSE},
year = {2026},
doi = {10.1145/3744916.3787837}
}