一句话：通过语义保持变换、重新划分数据与提出基于测试的基准，揭示微调AVR模型的过拟合与评测缺陷。
基于微调语言模型的自动漏洞修复（AVR）展现出潜力，但其修复未见漏洞的能力仍存疑。本文的实证研究发现：当前最优模型常过拟合训练集，且训练、验证与测试集并非互斥；基于 token 匹配的指标也难以反映多种有效补丁。作者从三方面重新评估：第一，对测试集施加语义保持变换，检验模型学到的是稳健修复模式还是虚假特征；第二，重新划分互斥数据集以评估泛化能力；第三，提出面向学习型 AVR 的基于测试的基准 L-AVRBench，以克服匹配指标局限并考察真实修复能力。
Learning-based automated vulnerability repair (AVR) techniques that utilize fine-tuned language models have shown promise in generating vulnerability patches. However, questions remain about their ability to repair unseen vulnerabilities. Our empirical study reveals that state-of-the-art models often overfit to the training set and are evaluated using training, validation, and test sets that are not mutually exclusive. Furthermore, relying on match-based metrics that compare generated patches to reference fixes at the token level has some limitations, failing to account for the possibility of various valid ways to patch the vulnerability. In this paper, we examine the capabilities of state-of-the-art fine-tuned AVR models and the adequacy of match-based evaluation metrics in three ways. First, we apply semantic-preserving transformations to test sets in order to determine whether models truly learn robust vulnerability-repair patterns or simply rely on spurious features. Second, we re-split the training, validation, and test sets to be mutually exclusive and evaluate the models on the revised test set to assess their generalization capabilities. Third, we introduce L-AVRBench, a test-based benchmark tailored for learning-based AVR, to overcome the limitations of match-based metrics and examine the AVR models’ true repair capabilities.
@inproceedings{abs-2512-22633,
title = {Rethinking the Capability of Fine-Tuned Language Models for Automated Vulnerability Repair},
author = {Woorim Han and Yeongjun Kwak and Miseon Yu and Kyeongmin Kim and Younghan Lee and Hyungon Moon and Yunheung Paek},
booktitle = {ICSE},
year = {2026},
doi = {10.1145/3744916.3773167}
}