一句话：在 SWE-bench Verified 上系统评测六个 LLM 缺陷修复智能体，剖析其能力边界与性能差异来源。
大语言模型与基于 LLM 的智能体已被用于自动修复缺陷，它们通过与环境交互、迭代验证和代码修改来应对软件缺陷，但目前尚缺乏对这些智能体系统的系统性分析，尤其是对表现最优系统之间性能差异的分析。本文在 SWE-bench Verified 基准上考察六个自动缺陷修复系统：首先评估各系统的整体性能，关注所有系统都能解决或都不能解决的实例，并探讨不同系统所具备的能力；其次在文件级与代码符号级上比较故障定位准确率，并评估缺陷复现能力。分析表明，要提升智能体修复缺陷的效果，既需要进一步优化 LLM 自身能力，也需要改进智能体流程（agentic flow）的设计。
Large language models (LLMs) and LLM-based Agents have been applied to fix bugs automatically, demonstrating the capability in addressing software defects by engaging in development environment interaction, iterative validation and code modification. However, systematic analysis of these agent systems remain limited, particularly regarding performance variations among top-performing ones. In this paper, we examine six repair systems on the SWE-bench Verified benchmark for automated bug fixing. We first assess each system’s overall performance, noting the instances solvable by all or none of these systems, and explore the capabilities of different systems. We also compare fault localization accuracy at file and code symbol levels and evaluate bug reproduction capabilities. Through analysis, we concluded that further optimization is needed in both the LLM capability itself and the design of Agentic flow to improve the effectiveness of Agents in bug fixing.