一句话：通过观察 19 名开发者与 IDE 内智能体协作修复 33 个 issue，揭示增量交互与迭代协作更易成功。
软件工程智能体（SWE agent）虽能在 SWE Bench 等基准上自主执行任务，但在复杂模糊的真实任务中仍面临困难，因此常被设计为可交互协作。本文通过观察 19 名开发者使用 IDE 内智能体解决其曾贡献仓库中的 33 个 open issue，研究开发者与智能体的协作方式及障碍。参与者成功解决约一半问题；采用增量方式者比一次性方式更成功；主动与智能体协作并迭代其输出者也更成功，但在信任智能体响应以及协作调试和测试方面仍遇挑战。研究建议智能体与开发者应在开发全流程中共同贡献，智能体应主动质疑和讨论，而非给出结论式或迎合式回应。
Software Engineering Agents (SWE agents) can autonomously perform development tasks on benchmarks like SWE Bench, but still face challenges when tackling complex and ambiguous real-world tasks. Consequently, SWE agents are often designed to allow interactivity with developers, enabling collaborative problem-solving. To understand how developers collaborate with SWE agents and the barriers they face in such interactions, we observed 19 developers using an in-IDE agent to resolve 33 open issues in repositories to which they had previously contributed. Participants successfully resolved about half of these issues, with those solving issues incrementally having greater success than those using a one-shot approach. Participants who actively collaborated with the agent and iterated on its outputs were also more successful, though they faced challenges in trusting the agent’s responses and collaborating on debugging and testing. Our findings suggest that to facilitate successful collaborations, both SWE agents and developers should actively contribute to tasks throughout all stages of the software development process. SWE agents can enable this by challenging and engaging in discussions with developers, rather than being conclusive or sycophantic.
DOI 原文 ·
@inproceedings{paperbot9572,
title = {Why AI Agents Still Need You: Findings from Developer-Agent Collaborations in the Wild},
author = {Aayush Kumar and Yasharth Bajpai and Sumit Gulwani and Gustavo Soares and Emerson Murphy-Hill},
booktitle = {ASE},
year = {2025},
doi = {10.1109/ase63991.2025.00043}
}