一句话:提出提示技术 μFiX,通过思维激发与反馈提示协同修正 LLM 对编程规约的误解,提升代码生成性能。
代码生成要求生成符合编程规约的源代码。LLM 生成的代码常与规约不一致;思维激发提示对复杂问题仍难形成正确理解,基于测试错误的反馈提示又因信息粗粒度而在代码严重偏离正确解时效果有限。本文提出提示技术 μFiX,首次探索思维激发与反馈提示的协同:先通过测试用例分析获得规约理解,并用自改进过程识别和修正思维激发阶段的误解;再在反馈阶段修正规约理解,以缩小显式提供理解与 LLM 生成代码时隐式使用的实际理解之间的差距。在 ChatGPT 和 DeepSeek-Coder 及六个常用基准上,与 15 个基线对比,μFiX 的 Pass@1 平均超过最强基线 35.62%。
Code generation is to automatically generate source code conforming to a given programming specification, which has received extensive attention especially with the development of large language models (LLMs). Due to the inherent difficulty of code generation, the code generated by LLMs may not be aligned with the specification. Although thought-eliciting prompting techniques have been proposed to enhance the code generation performance of LLMs, producing correct understanding for complicated programming problems remains challenging, resulting in unsatisfactory performance. Also, some feedbackbased prompting techniques have been proposed to fix incorrect code using error messages produced by test execution. However, when the generated code deviates significantly from the ground truth, they encounter difficulties in improving performance based on such coarse-grained information. In this work, we propose a novel prompting technique, called$\mu\mathbf{FiX}$, to improve the code generation performance of LLMs by devising both sophisticated thought-eliciting prompting and feedback-based prompting and making the first exploration on their synergy. It first exploits test case analysis to obtain specification understanding and enables a self-improvement process to identify and refine the misunderstanding in the thoughteliciting prompting phase.$\mu\mathbf{FiX}$further fixes the specification understanding towards the direction reducing the gap between the provided understanding (from the first phase) and the actual understanding implicity utilized by LLMs for code generation in the feedback-based prompting phase. By improving the understanding with$\mu \text{FiX}$, the code generation performance of LLMs can be largely improved. Our evaluation on two advanced LLMs (ChatGPT and DeepSeek-Coder) with six widely-used benchmarks by comparing with 15 baselines, demonstrates the effectiveness of$\mu\mathbf{FiX}$. For example,$\mu\mathbf{FiX}$outperforms the most effective baseline with an average improvement of 35.62 % in terms of Pass@1 across all subjects.
DOI 原文 ·
@inproceedings{paperbot8869,
title = {Fixing Large Language Models' Specification Misunderstanding for Better Code Generation},
author = {Zhao Tian and Junjie Chen and Xiangyu Zhang},
booktitle = {ICSE},
year = {2025},
doi = {10.1109/icse55347.2025.00108}
}