一句话:提出以代码注释为媒介、双向迭代对齐开发者意图与 LLM 生成的交互式代码生成方法。
LLM 生成代码常含功能性错误,复杂任务尤甚,开发者检查和修复困难,影响效率与信任。受通信中 Mutual Grounding 理论启发,本文提出交互式方法,以代码注释为开发者与 LLM 建立共享理解:迭代交错代码生成、行内注释生成和基于可编辑注释的情境化反馈,使代码对齐开发者意图。作者在两个基准上评估,方法显著提升多个 SOTA LLM,如 code-davinci-002 在 HumanEval 上 pass@1 提升 17.1%;12 人用户研究对比 GitHub Copilot 和 Multi-Turn Program Synthesis,任务完成快 16.7%,成功率提升 10.5%。
Large Language Models (LLMs) have demonstrated unprecedented capability in code generation. However, LLM-generated code is still plagued with a wide range of functional errors, especially for complex programming tasks that LLMs have not seen before. Recent studies have shown that developers often struggle with inspecting and fixing incorrect code generated by LLMs, diminishing their productivity and trust in LLM-based code generation. Inspired by the mutual grounding theory in communication, we propose an interactive approach that leverages code comments as a medium for developers and LLMs to establish a shared understanding. Our approach facilitates iterative grounding by interleaving code generation, inline comment generation, and contextualized user feedback through editable comments to align generated code with developer intent. We evaluated our approach on two popular benchmarks and demonstrated that our approach significantly improved multiple state-of-the-art LLMs, e.g., 17.1% pass@1 improvement for code-davinci-002 on HumanEval. Furthermore, we conducted a user study with 12 participants in comparison to two baselines: (1) interacting with GitHub Copilot, and (2) interacting with a multi-step code generation paradigm called Multi-Turn Program Synthesis. Participants completed the given programming tasks 16.7% faster and with 10.5% improvement in task success rate when using our approach. Both results show that interactively refining code comments enables the collaborative establishment of mutual grounding, leading to more accurate code generation and higher developer confidence.
DOI 原文 ·
@inproceedings{paperbot8828,
title = {Enhancing Code Generation via Bidirectional Comment-Level Mutual Grounding},
author = {Yifeng Di and Tianyi Zhang},
booktitle = {ICSE},
year = {2025},
doi = {10.1109/icse55347.2025.00165}
}