一句话：提出 D4C 提示框架，通过对齐 LLM 训练目标并直接调试整个程序来修复缺陷，大幅减少采样。
当前基于大语言模型的自动程序修复（APR）面临两个错配：一是 decoder-only LLM 的下一词预测训练目标与现有 infilling 式方法的掩码片段预测目标不一致，阻碍模型充分利用预训练知识；二是现有方法仍依赖语句级故障定位提供候选缺陷代码块，限制了 LLM 探索给定位置之外的补丁。本文提出 D4C——一个直接的提示框架：让 LLM 的输出形式与其训练目标对齐，并允许其在不预先定位故障语句的情况下直接细化整个程序。在 Defects4J 上，每个补丁仅采样 10 次，D4C 即正确修复 180 个缺陷，比采用完美故障定位的 SOTA APR 方法高出 10%，同时将补丁采样数量减少 90%。结果表明，目标对齐对释放 LLM 预训练能力至关重要，且以直接调试取代“先定位缺陷块再修复”的传统流程，对基于 LLM 的 APR 更有效。
Large language models (LLMs) have achieved decent results on automated program repair (APR). However, the next token prediction training objective of decoder-only LLMs (e.g., GPT-4) is misaligned with the masked span prediction objective of current infilling-style methods, which impedes LLMs from fully leveraging pre-trained knowledge for program repair. In addition, while some LLMs can locate and repair bugs in certain functions using the related artifacts (e.g., test cases), existing methods still depend on statement-level fault localization methods to provide a list of buggy hunks for repair. This restriction hinders LLMs from exploring potential patches beyond the given locations. In this paper, we investigate a new approach to adapt LLMs to program repair. Our core insight is that LLM's APR capability can be greatly improved by simply aligning the output to their training objective and allowing them to refine the whole program without first identifying faulty statements. Based on this insight, we designed D4C, a straightforward prompting framework for APR. D4C can repair 180 bugs correctly in Defects4J, with each patch being sampled only 10 times. This surpasses the SOTA APR methods with perfect fault localization by 10 % and reduces the patch sampling number by 90 %. Our findings reveal that (1) objective alignment is crucial for fully exploiting LLM's pre-trained capability, and (2) replacing the traditional localize-buggy-hunks-then-repair workflow with direct debugging is more effective for LLM-based APR methods. Thus, we believe this paper introduces a new mindset for harnessing LLMs in APR.
DOI 原文 ·
@inproceedings{paperbot8705,
title = {Aligning the Objective of LLM-Based Program Repair},
author = {Junjielong Xu and Ying Fu and Shin Hwei Tan and Pinjia He},
booktitle = {ICSE},
year = {2025},
doi = {10.1109/icse55347.2025.00169}
}