paperbot · PL 论文追踪

全部论文 待读 收藏

P2 Heterogeneous Prompting and Execution Feedback for SWE Issue Test Generation and Selection

用于 SWE issue 测试生成与选择的异构提示与执行反馈
ICSE Proceedings of the 2026 IEEE/ACM 48th International Conference on Software Engineering2026匹配度 3/10 · 该论文属于 LLM 用于软件工程测试生成与选择,虽涉及执行反馈和 SWE issue,但不涉及静态分析、形式化验证、浮点程序分析或贝叶斯程序分析,与用户核心方向交集有限。其贡献也偏向提示技巧与测试生成,而非用户关注的 PL 与 AI 交叉分析。引用 13 inbox
Toufique Ahmed, Jatin Ganhotra, Avraham Shinnar, Martin Hirzel
crossref: 2026-09-12arxiv: 2026-09-12

AI 速览

一句话:提出异构提示与执行反馈技术,自动为软件工程 issue 生成并选择复现测试。

许多软件工程 issue 缺少可运行的复现测试,阻碍问题定位与修复。本文研究如何自动生成此类测试。针对温度采样推理扩展多样性不足的问题,提出异构提示;针对 issue 解决前新代码尚不存在、难以利用执行反馈的问题,提出新的执行反馈途径。作者在 e-Otter++ 中实现这些技术,并在 TDD-Bench Verified 基准上评估,生成的测试平均 fail-to-pass 率达 63%,显著推进了该任务的最优水平。

问题
大多数软件工程 issue 不附带可运行的复现测试,如何自动生成能暴露问题的复现测试是核心问题。
方法
提出异构提示以提升推理扩展的多样性,并设计在修复代码尚不存在时获取执行反馈的新方法,集成到复现测试生成器 e-Otter++ 中。
评估
在 TDD-Bench Verified 基准上评估,e-Otter++ 生成的测试平均 fail-to-pass 率为 63%,超过现有最优方法。
原文摘要(Abstract)

A software engineering issue (SWE issue) is easier to resolve when accompanied by a reproduction test. Unfortunately, most issues do not come with functioning reproduction tests, so this paper explores how to generate them automatically. One technique that helps with this task is inference scaling, but traditional temperature-based scaling tends to lack diversity. This paper introduces heterogeneous prompting to address this problem. Another technique that helps with this task is execution feedback, but this is hampered by the fact that the new code (after issue resolution) to execute does not yet exist. This paper introduces novel approaches to get around that problem. We implemented our techniques in a new reproduction test generator called e-Otter++. Experiments show that e-Otter++ represents a leap ahead in the state-of-the-art for this problem, generating tests with an average fail-to-pass rate of 63% on the TDD-Bench Verified benchmark.

链接与引用

DOI 原文 · arXiv ·

BibTeX
@inproceedings{paperbot11827,
  title = {Heterogeneous Prompting and Execution Feedback for SWE Issue Test Generation and Selection},
  author = {Toufique Ahmed and Jatin Ganhotra and Avraham Shinnar and Martin Hirzel},
  booktitle = {ICSE},
  year = {2026},
  doi = {10.1145/3744916.3787837}
}