一句话:用领域推理数据与课程式可验证奖励强化学习训练推理 LLM,实现自动化漏洞修复。
基于 LLM 的自动漏洞修复(AVR)虽性能领先,却受限于两点:缺乏高质量的漏洞相关推理数据,难以捕捉多样修复模式;修复过程缺少可验证的中间反馈,强化学习训练困难。作者将漏洞修复建模为推理任务,提出推理 LLM Vul-R2:一是领域感知推理学习模块,构造并过滤推理答案后监督微调,注入漏洞推理知识;二是课程式可验证奖励训练模块,简单阶段用多选题问答、困难阶段用字符级匹配作为可验证奖励,动态强化学习。在真实 C/C++ 数据集 PrimeVul 上,EM 比最佳基线高 11.27%,多修复 49 个漏洞;仅在该数据集微调,即在人工标注集 SVEN 上提升 8.78% EM。
The exponential increase in software vulnerabilities has created an urgent need for automatic vulnerability repair (AVR) solutions. Recent research has formulated AVR as a sequence generation problem and has leveraged large language models (LLMs) to address this problem. Typically, these approaches prompt or fine-tune LLMs to generate repairs for vulnerabilities directly. Although these methods show state-of-the-art performance, they face the following challenges: (1) Lack of high-quality, vulnerability-related reasoning data. Current approaches primarily rely on foundation models that mainly encode general programming knowledge. Without vulnerability-related reasoning data, they tend to fail to capture the diverse vulnerability repair patterns. (2) Hard to verify the intermediate vulnerability repair process during LLM training. Existing reinforcement learning methods often leverage intermediate execution feedback from the environment (e.g., sandbox-based execution results) to guide reinforcement learning training. In contrast, the vulnerability repair process generally lacks such intermediate, verifiable feedback, which poses additional challenges for model training.To address these challenges, we propose to model the vulnerability repair task from a reasoning perspective and train a reasoning LLM termed Vulnerability Reasoner and Repair (Vul-R2) which consists of two key modules: (1) a domain-aware reasoning learning module, which comprises a reasoning answer construction component, a reasoning data filtering process, and a supervised fine-tuning process for learning vulnerability-related reasoning knowledge; and (2) a curriculum-based verifiable rewarded training module, which comprises dynamically reinforcement learning with verifiable rewards paradigms based on multiple-choice question answering in an easy stage and character-level matching in a hard stage. We evaluate Vul-R2 on the real-world C/C++ dataset PrimeVul to demonstrate its effectiveness in vulnerability repair. Specifically, Vul-R2 outperforms the best baseline by 11.27% for exact match (EM) and successfully repairs 49 additional vulnerabilities. Furthermore, we demonstrate the effectiveness of the proposed paradigm, fine-tuning Vul-R2 on PrimeVul leads to improved EM performance of 8.78% on a human curated dataset SVEN, even without additional training.
DOI 原文 ·
@inproceedings{paperbot9240,
title = {Vul-R2: A Reasoning LLM for Automated Vulnerability Repair},
author = {Xin-Cheng Wen and Zirui Lin and Yijun Yang and Cuiyun Gao and Deheng Ye},
booktitle = {ASE},
year = {2025},
doi = {10.1109/ase63991.2025.00011}
}