paperbot · PL 论文追踪

全部论文 待读 收藏

P2 Fixing Large Language Models' Specification Misunderstanding for Better Code Generation

修复大语言模型对规约的误解以改进代码生成
ICSE 2025 IEEE/ACM 47th International Conference on Software Engineering (ICSE)2025匹配度 2/10 · 论文属于 LLM 代码生成的提示工程,核心是改进 LLM 对编程规约的理解,未涉及静态分析、形式化验证、浮点或贝叶斯程序分析等用户主要方向。且研究目标本身是提升 LLM 能力,属于用户明确不感兴趣的提示词技巧类工作。引用 7 inbox
Zhao Tian, Junjie Chen, Xiangyu Zhang
crossref: 2026-09-12

AI 速览

一句话:提出提示技术 μFiX,通过思维激发与反馈提示协同修正 LLM 对编程规约的误解,提升代码生成性能。

代码生成要求生成符合编程规约的源代码。LLM 生成的代码常与规约不一致;思维激发提示对复杂问题仍难形成正确理解,基于测试错误的反馈提示又因信息粗粒度而在代码严重偏离正确解时效果有限。本文提出提示技术 μFiX,首次探索思维激发与反馈提示的协同:先通过测试用例分析获得规约理解,并用自改进过程识别和修正思维激发阶段的误解;再在反馈阶段修正规约理解,以缩小显式提供理解与 LLM 生成代码时隐式使用的实际理解之间的差距。在 ChatGPT 和 DeepSeek-Coder 及六个常用基准上,与 15 个基线对比,μFiX 的 Pass@1 平均超过最强基线 35.62%。

问题
LLM 生成的代码常与给定编程规约不一致;现有提示方法对复杂问题的规约理解不足,基于测试错误的反馈提示在代码严重偏离正确解时改进有限。
方法
提出 μFiX,将思维激发提示与反馈提示协同:先用测试用例分析获得规约理解并自改进修正误解,再基于反馈修正规约理解以缩小显式理解与 LLM 隐式理解之间的差距。
评估
在 ChatGPT 与 DeepSeek-Coder 两个 LLM、六个常用基准上,与 15 个基线比较,按 Pass@1 评估,平均比最强基线提升 35.62%。
原文摘要(Abstract)

Code generation is to automatically generate source code conforming to a given programming specification, which has received extensive attention especially with the development of large language models (LLMs). Due to the inherent difficulty of code generation, the code generated by LLMs may not be aligned with the specification. Although thought-eliciting prompting techniques have been proposed to enhance the code generation performance of LLMs, producing correct understanding for complicated programming problems remains challenging, resulting in unsatisfactory performance. Also, some feedbackbased prompting techniques have been proposed to fix incorrect code using error messages produced by test execution. However, when the generated code deviates significantly from the ground truth, they encounter difficulties in improving performance based on such coarse-grained information. In this work, we propose a novel prompting technique, called$\mu\mathbf{FiX}$, to improve the code generation performance of LLMs by devising both sophisticated thought-eliciting prompting and feedback-based prompting and making the first exploration on their synergy. It first exploits test case analysis to obtain specification understanding and enables a self-improvement process to identify and refine the misunderstanding in the thoughteliciting prompting phase.$\mu\mathbf{FiX}$further fixes the specification understanding towards the direction reducing the gap between the provided understanding (from the first phase) and the actual understanding implicity utilized by LLMs for code generation in the feedback-based prompting phase. By improving the understanding with$\mu \text{FiX}$, the code generation performance of LLMs can be largely improved. Our evaluation on two advanced LLMs (ChatGPT and DeepSeek-Coder) with six widely-used benchmarks by comparing with 15 baselines, demonstrates the effectiveness of$\mu\mathbf{FiX}$. For example,$\mu\mathbf{FiX}$outperforms the most effective baseline with an average improvement of 35.62 % in terms of Pass@1 across all subjects.

链接与引用

DOI 原文 ·

BibTeX
@inproceedings{paperbot8869,
  title = {Fixing Large Language Models' Specification Misunderstanding for Better Code Generation},
  author = {Zhao Tian and Junjie Chen and Xiangyu Zhang},
  booktitle = {ICSE},
  year = {2025},
  doi = {10.1109/icse55347.2025.00108}
}