paperbot · PL 论文追踪

全部论文 待读 收藏

Optimization-Free Repair of Code LLMs: A Closed-Form Fisher-Guided Bayesian Approach

代码 LLM 的无优化修复:一种闭式 Fisher 引导的贝叶斯方法
ASE Proceedings of the 41st IEEE/ACM International Conference on Automated Software Engineering2026引用 0 inbox
Zhiyu Duan, Chong Bian, Yizhuo Zhang, Shunkun Yang
crossref: 2026-10-09

AI 速览

一句话:提出 FISR,用 Fisher 信息矩阵引导的闭式贝叶斯后验更新修复代码 LLM,兼顾准确、稳定与高效。

大型语言模型生成的代码常含细微功能错误,现有修复方法进行局部参数更新时未考虑预训练知识的重要性差异,容易损害通用编码能力。本文提出 Fisher 引导语义修复(FISR),将代码 LLM 修复视为受预训练先验约束的贝叶斯后验更新。它通过 Fisher 信息矩阵估计参数级语义重要性,利用局部曲率识别应保留的高置信方向,并引入 Fisher 加权结构约束,使修正更新避开知识关键参数、转向更可塑区域。结合结构化曲率近似与低秩精度分解,该方法在安全修复子空间内实现高效闭式更新,无需迭代重训练。在三个代码骨干模型和六个软件工程基准上,FISR 在修复准确率、稳定性与效率间取得最佳权衡,Exact Match 最高达 95.8%,较强基线提升最多 3.1%,Pass@1 提升最多 3.4%,端到端修复时间仅为竞争编辑基线的 40%-67%,更新阶段耗时仅 14%-20%。

问题
代码 LLM 的生成结果易含细微功能错误,而现有参数更新式修复未区分预训练知识的重要性,可能将针对性修正与通用编码能力破坏纠缠在一起。
方法
将修复建模为受预训练先验约束的贝叶斯后验更新,利用 Fisher 信息矩阵与低秩精度分解,在安全修复子空间内进行闭式参数修正,避免迭代重训练。
评估
在三个代码骨干模型和六个软件工程基准上,与强基线比较 Exact Match、Pass@1、留出稳定性及端到端与更新阶段耗时。
原文摘要(Abstract)

Large language models (LLMs) are increasingly used in software engineering, yet their code generations remain vulnerable to subtle functional errors, making reliable post-hoc repair essential. Existing repair methods typically apply localized parameter updates without explicitly accounting for the heterogeneous importance of pre-trained knowledge, which can entangle targeted correction with unintended changes to general coding ability. We propose Fisher-Informed Semantic Repair (FISR), a principled framework that treats code LLM repair as a Bayesian posterior update constrained by pre-trained prior knowledge. It estimates parameter-wise semantic importance through the Fisher Information Matrix, using local curvature information to identify high-confidence directions that should be preserved during repair. It then introduces Fisher-weighted structural constraints to guide corrective updates away from knowledge-critical parameters and toward more adaptable regions of the parameter space. To make this process scalable, we combine structured curvature approximation with low-rank precision decomposition, enabling efficient closed-form updates within a safe repair subspace rather than relying on iterative retraining. Extensive experiments on three backbone code models and six software engineering benchmarks show that FISR achieves the best overall trade-off among repair accuracy, stability, and efficiency. It reaches up to 95.8% Exact Match, improving Exact Match by up to 3.1% and Pass@1 by up to 3.4% over strong prior repair methods. It also demonstrates stronger held-out stability while substantially reducing repair cost, requiring only 40%-67% of the end-to-end repair time and 14%-20% of the update-time cost of competitive editing-based baselines. These results demonstrate that the proposed method enables accurate, safe, and efficient semantic repair for code LLMs.

链接与引用

DOI 原文 · PDF(开放获取) ·

BibTeX
@inproceedings{paperbot12241,
  title = {Optimization-Free Repair of Code LLMs: A Closed-Form Fisher-Guided Bayesian Approach},
  author = {Zhiyu Duan and Chong Bian and Yizhuo Zhang and Shunkun Yang},
  booktitle = {ASE},
  year = {2026},
  doi = {10.1145/3832783.3834442}
}