一句话:系统研究 LLM 问题修复智能体的失败模式,并提出类型感知的后处理代理 PAGENT 修复错误补丁。
该论文关注基于大语言模型的问题修复智能体在真实软件缺陷上生成的补丁仍大量错误或不完整,且失败原因缺乏系统理解的问题。作者在 SWE-Bench Lite 上对七种先进修复智能体进行实证研究,聚焦所有智能体均失败的 114 个问题和 798 个失败补丁,结合 LLM 驱动分析与专家人工审查,归纳出五类反复出现的失败模式,其中类型与数据结构管理不当占全部失败的 27.19%。基于此,论文提出模型无关的后处理代理 PAGENT,采用混合架构整合仓库级静态分析和基于 LLM 的类型推断,以修复生成补丁中的类型相关错误。在七个智能体上,PAGENT 成功修复 217 个类型相关失败中的 75 个,占 34.56%。结果表明当前 LLM 修复智能体存在根本局限,而针对性的类型感知后处理能显著挽回在困难真实问题上的性能损失。
Large language model (LLM)-based issue resolution agents can automatically propose patches for software bugs, yet a substantial portion of their patches remain incorrect or incomplete, and the root causes of these failures are still poorly understood. In this work, we conduct a systematic empirical study of seven state-of-the-art issue resolution agents on SWE-Bench Lite, focusing on 114 issues that all agents failed to resolve and 798 associated failed patches. Using a combined pipeline of LLM-driven analysis and expert manual review, we derive a taxonomy of five recurrent failure patterns, showing that type and data-structure mismanagement alone account for 27.19% of all failures. Motivated by these insights, we introduce PAGENT, a model-agnostic post-processing agent that augments existing systems with a hybrid architecture integrating repository-level static analysis and LLM-based type inference to repair type-related faults in generated patches. Across all seven agents, PAGENT successfully repairs 75 out of 217 type-related failures (34.56%). Our findings both characterize fundamental limitations of current LLM-based repair agents and demonstrate that targeted, type-aware post-processing can substantially recover otherwise lost performance on challenging, real-world issues.