一句话：提出 STAR，将 LLM 修复重构为语义驱动的优化过程，定位并修补故障神经元，在代码生成任务上显著超越现有方法。
大语言模型在代码生成中常产生错误代码，相比只修复输出，修复模型本身更彻底，但完整重训练代价高昂。语言模型修复（LM repair）只需少量数据和较小算力，副作用有限，然而先前方法 MINT 与标准优化方法 SGD 各有局限。本文提出 STAR（Semantic Targeting for Analytical Repair），将 LM 修复的关键操作统一到一个优化过程中：定位「故障神经元」、求解「神经元补丁」并完成修补。神经元补丁由基于语义的解析公式计算，通过引导隐表示，直接将 logits 的变化与神经元增量联系起来，从而兼具 MINT 与 SGD 的优点并缓解其不足；该重构还允许一次性解决多个失败，提升实用性。在流行代码 LM 的编码任务上，STAR 优于当前最优方法：在 CoNaLa 上 ExactMatch 提升 10.5%–19.9%，求解每个模型失败平均加速 2.4–7.0 倍，且在泛化性与特异性的平衡上明显更优。作者还评估了修复的过拟合风险与累积影响，并分析了其与流水线式方法的差异及缓解常见局限的原因。
Language Models (LMs) are widely used in software engineering for code generation, but they may produce erroneous code. Rather than repairing outputs, a more thorough remedy is to address underlying model failures. LM repair offers a lightweight solution: it requires minimal data, lowers computational cost, and limits side effects. Unlike full retraining, LM repair focuses on applying tailored updates to targeted neurons, making it suitable for limited resources, high-performance demands, or strict safety requirements. In this paper, we propose Semantic Targeting for Analytical Repair (STAR), a novel semantic-based optimization method for repairing LLMs. STAR realizes the main operations of repairing LMs in an optimization process, including locating “buggy neurons”, solving “neuron patches”, and patching “buggy neurons”. The neuron patches are computed with a solid semantic-based analytical formula, which directly bridges the changes to logits with the deltas of neurons, by steering latent representations. Compared to the prior work of LM repair (MINT) and standard optimization methods (SGD), STAR integrates their strengths while mitigating their limitations. By reformulating LM repair as an optimization process, STAR may solve multiple failures together, significantly improving the usefulness. Evaluated on coding tasks using popular code LMs, STAR demonstrates superior effectiveness compared with the state-of-the-art. For instance, on CoNaLa, STAR obtained 10.5%–19.9% improvements in ExactMatch score. Besides, STAR exhibits better efficiency, achieving an average of 2.4–7.0 times speedup in solving each model failure. In terms of side effects, namely the balance between generalization and specificity, STAR outperforms prior work by a significant margin. Additionally, we conducted assessments on the overfitting risk of LM repair as well as the cumulative impact. Further, we analyzed the differences with pipeline-based methods and explained the reason why STAR is better and how it mitigated the common limitations of LM repair.
@inproceedings{abs-2503-12899,
title = {A Semantic-based Optimization Approach for Repairing LLMs: Case Study on Code Generation},
author = {Jian Gu and Aldeida Aleti and Chunyang Chen and Hongyu Zhang},
booktitle = {ICSE},
year = {2026},
doi = {10.1145/3744916.3773262}
}