一句话：提出故障引导微调 FGit，通过提取正确与错误实现的细粒度差异并动态加权，提升 LLM 代码生成正确率。
指令微调 LLM 在代码生成上进展显著，但标准监督微调常生成看似合理却功能错误的代码，且未能强调错误敏感片段。本文提出故障引导微调 FGit：先提取正确实现与相似错误变体之间的行级/词元级多粒度差异，定位错误敏感片段；再通过动态损失加权在训练中优先这些片段。作者在 7 个 LLM 和 3 个常用基准上实验，pass@1 平均相对提升 6.9%，部分增强的 6.7B 模型超过 GPT-3.5-Turbo；跨模型泛化提升 3.8%–19.1%，消融验证了不同粒度差异及超参数的作用。
Modern instruction-tuned large language models (LLMs) have made remarkable progress in code generation. However, these LLMs fine-tuned with standard supervised fine-tuning (SFT) sometimes generate plausible-looking but functionally incorrect code fails to emphasize the error-sensitive segments—specific code differences between correct implementations and similar incorrect variants. To address this problem, we propose Fault-Guided Fine-Tuning (FGit), a novel fine-tuning technique that enhances LLMs’ code generation by (1) extracting multi-granularity (line/token-level) differences between correct and incorrect yet similar implementations to identify error-sensitive segments, and (2) dynamically prioritizing those segments during training via dynamic loss weighting. Through extensive experiments on seven LLMs across three widely-used benchmarks, our method achieves an average relative improvement of 6.9% on pass@1, with some enhanced 6.7B LLMs outperforming closed-source models, e.g., GPT-3.5-Turbo. Furthermore, our fine-tuning technique demonstrates strong generalization with performance improvements ranging from 3.8% to 19.1% across diverse instruction-tuned LLMs, and our ablation studies confirm the contributions of different granularities of differences and hyperparameters.
DOI 原文 ·
@inproceedings{paperbot9254,
title = {FGit: Fault-Guided Fine-Tuning for Code Generation},
author = {Lishui Fan and Zhongxin Liu and Haoye Wang and Lingfeng Bao and Xin Xia and Shanping Li},
booktitle = {ASE},
year = {2025},
doi = {10.1109/ase63991.2025.00114}
}