一句话：提出统一测试时计算扩展框架，使32B开源模型在SWE-bench Verified上达到46%解决率，超越更大模型。
软件工程智能体在自动化程序改进上展现出潜力，但其依赖闭源或资源密集型模型，导致私有环境部署困难，核心问题是：单卡可部署的开源LLM（如32B模型）如何达到可比的代码推理性能？为此，本文提出统一的测试时计算（TTC）扩展框架，用增加推理期计算替代增大模型规模。框架包含内部与外部两种互补策略：内部提出开发情境化轨迹合成方法，利用真实软件仓库引导故障定位、补丁生成等多阶段推理，并通过拒绝采样按准确性与复杂度严格筛选轨迹；外部提出基于开发过程的搜索策略，由奖励模型与执行验证引导，在关键开发决策点定向分配计算，克服现有仅端点验证的局限。在SWE-bench Verified上，32B模型取得46%的问题解决率，超过DeepSeek R1 671B与OpenAI o1等更大模型；还实证验证了SWE智能体中的测试时扩展现象，模型会为更难问题动态分配更多token，有效增强推理能力。训练数据、模型与代码均已公开，方法已部署于通义灵码。
Recent advancements in software engineering agents have demonstrated promising capabilities in automating program improvements. However, their reliance on closed-source or resource-intensive models introduces significant deployment challenges in private environments, prompting a critical question: How can personally deployable open-source LLMs (e.g., 32B models running on a single GPU) achieve comparable code reasoning performanceƒ To this end, we propose a unified Test-Time Compute (TTC) scaling framework that leverages increased inference-time computation instead of larger models. Our framework incorporates two complementary strategies: internal TTC and external TTC. Internally, we introduce a development-contextualized trajectory synthesis method leveraging real-world software repositories to bootstrap multi-stage reasoning processes, such as fault localization and patch generation. We further enhance trajectory quality through rejection sampling, rigorously evaluating trajectories along accuracy and complexity. Externally, we propose a novel development-process-based search strategy guided by reward models and execution verification. This approach enables targeted computational allocation at critical development decision points, overcoming limitations of existing "end-point only" verification methods.Evaluations on SWE-bench Verified demonstrate our 32B model achieves a 46% issue resolution rate, surpassing significantly larger models such as DeepSeek R1 671B and OpenAI o1. Additionally, we provide the empirical validation of the test-time scaling phenomenon within SWE agents, revealing that models dynamically allocate more tokens to increasingly challenging problems, effectively enhancing reasoning capabilities. We publicly release all training data, models, and code to facilitate future research.1. In fact, our method has been deployed in Tongyi Lingma, an IDE-based coding assistant developed by Alibaba Cloud, where it helps developers solve real-world programming problems.
DOI 原文 ·
@inproceedings{paperbot9231,
title = {Thinking Longer, Not Larger: Enhancing Software Engineering Agents via Scaling Test-Time Compute},
author = {Yingwei Ma and Yongbin Li and Yihong Dong and Xue Jiang and Yanhao Li and Yue Liu and Rongyu Cao and Jue Chen and Fei Huang and Binhua Li},
booktitle = {ASE},
year = {2025},
doi = {10.1109/ase63991.2025.00309}
}