paperbot · PL 论文追踪

全部论文 待读 收藏

Free Is More: Empowering Software Agents via Issue-Free Trajectory Learning and Entropy-Aware RLVR Training

Free Is More:通过无 issue 轨迹学习与熵感知 RLVR 训练增强软件智能体
ASE Proceedings of the 41st IEEE/ACM International Conference on Automated Software Engineering2026引用 0 inbox
Xin-Cheng Wen, Binbin Chen, Haoxuan Lan, Hang Yu, Peng Di, Cuiyun Gao
crossref: 2026-10-09

AI 速览

一句话:提出 SWE-Fuse,用无 issue 轨迹微调与熵感知裁剪 RLVR 提升软件智能体修复真实问题的能力。

面向真实软件 issue 修复的 LLM 智能体通常配备编码工具并自主生成动作轨迹,但面临两个问题:issue 描述与解决方案不一致会引入噪声,且高质量 issue-PR 数据稀缺;同时 bash 工具调用带来高不确定性,使 RLVR 中固定的概率比裁剪难以兼顾探索与稳定训练。为此,作者提出 SWE-Fuse 训练框架,融合 issue 描述引导与无 issue 样本:其无 issue 驱动轨迹学习模块构建高质量多步推理-动作轨迹、过滤轨迹数据并进行无 issue 监督微调;熵感知 RLVR 模块则依熵自适应裁剪,高熵时放松以鼓励探索,低熵时收紧以稳定训练。在 SWE-bench Verified 上,SWE-Fuse 使 8B 与 32B 模型的 solve rate 分别较最佳基线提升 41.9% 与 59.3%,结合 TTS@8 后达到 49.8% 与 65.2%。

问题
LLM 软件智能体在真实 issue 修复中受低质量/不一致 issue 描述干扰,且 bash 工具带来的高不确定性使 RLVR 固定裁剪机制难以稳定训练并充分探索。
方法
提出 SWE-Fuse,融合无 issue 轨迹学习与熵感知 RLVR:构建并过滤多步推理-动作轨迹进行无 issue 监督微调,同时按熵自适应放松或收紧概率比裁剪。
评估
在 SWE-bench Verified 基准上评估 8B 与 32B 模型,并与最佳基线比较 solve rate,同时测试与 test-time scaling(TTS@8)结合后的效果。
原文摘要(Abstract)

Large language models (LLMs) have transformed the software engineering landscape. Recently, numerous LLM-based agents have been developed to address real-world software issue fixing tasks. These agents are typically equipped with various coding tools, enabling them to autonomously generate action trajectories for end-to-end problem-solving. Despite their state-of-the-art performance, these agents still face two primary challenges: (1) Lack of sufficiently high-quality issue descriptions: Inconsistencies between issue descriptions and their corresponding solutions introduce noise and ambiguity that can mislead automated agents, restricting their ability to derive effective solutions. Furthermore, high-quality issue-PR pairs are difficult to acquire at scale, and the available datasets remain relatively limited in size. (2) Uncertainty introduced by bash tools prevents stable training and sufficient exploration: Bash tools serve as the primary interface for code execution and file manipulation in SWE agents, making them integral to virtually all agent operations. However, bash tools introduce substantial uncertainty into the agent's decision-making process. In reinforcement learning with verifiable rewards (RLVR), the fixed probability ratio clipping mechanism exhibits limited adaptability to such varying uncertainty states. Specifically, when bash tool invocations trigger high-entropy states, excessively restrictive clipping constraints impede both exploration and stable learning, hindering the training efficiency. To address these challenges, we propose SWE-Fuse, an issue-description-aware training framework that fuses issue-description-guided and issue-free samples for training SWE agents. It consists of two key modules: (1) An issue-free-driven trajectory learning module, which comprises: a multi-step trajectory construction component for generating high-quality multi-turn reasoning-action trajectories, a trajectory data filtering process for ensuring data quality, and issue-free-driven supervised fine-tuning to mitigate potentially misleading issue descriptions while enabling the model to learn step-by-step debugging processes; and (2) An entropy-aware RLVR training module, which adaptively adjusts training dynamics through entropy-driven clipping. It applies relaxed clipping under high entropy to encourage exploration, and stricter clipping under low entropy to ensure training stability. We evaluate SWE-Fuse on the widely studied SWE-bench Verified benchmark shows to demonstrate its effectiveness in solving real-world software problems. Specifically, SWE-Fuse outperforms the best 8B and 32B baselines by 41.9% and 59.3% in solve rate, respectively. Furthermore, integrating SWE-Fuse with test-time scaling (TTS) enables further performance improvements, achieving solve rates of 49.8% and 65.2% under TTS@8 for the 8B and 32B models, respectively.

链接与引用

DOI 原文 · PDF(开放获取) ·

BibTeX
@inproceedings{paperbot12207,
  title = {Free Is More: Empowering Software Agents via Issue-Free Trajectory Learning and Entropy-Aware RLVR Training},
  author = {Xin-Cheng Wen and Binbin Chen and Haoxuan Lan and Hang Yu and Peng Di and Cuiyun Gao},
  booktitle = {ASE},
  year = {2026},
  doi = {10.1145/3832783.3834438}
}