一句话：重新审视程序修复中的非自然性，用熵度量补足 LLM，提升故障定位、补丁生成与分类效果。
大语言模型（LLM）在代码生成上的进步为自动程序修复（APR）带来新机会，但因训练语料中缺陷稀少，模型倾向把缺陷代码视为低概率（非自然），同时生成更自然的正确代码。本文重新审视程序修复中的（非）自然性，主张 LLM 单独修复有局限，需引入压缩上下文与分析信息的传统工具。作者考察传统修复各阶段的熵，将自然性度量与频谱故障定位（SBFL）结合，使 Top-5 评分较 SBFL 提升 50%；用熵差指导补丁生成，平均每次修复减少 24 次测试执行，并有效区分正确补丁与过拟合补丁。结果表明 LLM 可有效补充经典分析与变换技术。
The problem of software quality has motivated the development of a variety of techniques for Automatic Program Repair (APR). Meanwhile, recent advances in AI and Large Language Models (LLMs) have produced orders of magnitude performance improvements over previous code generation techniques, affording promising opportunities for program repair and its constituent subproblems (e.g., fault localization, patch generation). Because models are trained on large volumes of code in which defects are relatively rare, they tend to both simultaneously perceive faulty code as unlikely (or “unnatural”) and to produce generally correct code (which is more “natural”). This paper comprehensively revisits the idea of (un)naturalness for program repair. We argue that, fundamentally, LLMs can only go so far on their own in reasoning about and fixing buggy code. This motivates the incorporation of traditional tools, which compress useful contextual and analysis information, as a complement to LLMs for repair. We interrogate the role of entropy at every stage of traditional repair, and show that it is indeed usefully complementary to classic techniques. We show that combining measures of naturalness with class Spectrum-Based Fault Localization (SBFL) approaches improves Top-5 scoring by 50 % over SBFL alone. We show that entropy delta, or change in entropy induced by a candidate patch, can improve patch generation efficiency by 24 test suite executions per repair, on average, on our dataset. Finally, we show compelling results that entropy delta for patch classification is highly effective at distinguishing correct from overfitting patches. Overall, our results suggest that LLMs can effectively complement classic techniques for analysis and transformation, producing more efficient and effective automated repair techniques overall.
DOI 原文 ·
@inproceedings{paperbot8706,
title = {Revisiting Unnaturalness for Automated Program Repair in the Era of Large Language Models},
author = {Aidan Z.H. Yang and Sophia Kolak and Vincent Hellendoorn and Ruben Martins and Claire Le Goues},
booktitle = {ICSE},
year = {2025},
doi = {10.1109/icse55347.2025.00089}
}