一句话:提出 Specine,通过识别、提升并对齐编程规约,缓解 LLM 代码生成与需求不一致的问题。
代码生成指依据给定编程规约自动产生源代码,随着大语言模型的发展受到广泛关注。然而真实问题固有的复杂性使 LLM 生成的代码往往无法完全符合给定规约;现有基于 agent 的增强技术忽略了规约感知这一关键问题,导致错位持续存在。鉴于准确感知编程规约是基于 LLM 的代码生成范式的基础,保证规约对齐尤为重要。本文借鉴软件需求工程的思想,提出 Specine——一种面向 LLM 代码生成的规约对齐技术,其核心思路是识别错位的输入规约、提升 LLM 所感知的规约,并对其进行对齐以增强 LLM 的代码生成表现。作者在四个先进 LLM 与五个具有挑战性的竞赛基准上,与十个先进基线进行了全面实验,验证了 Specine 的有效性:例如在 Pass@1 指标上,Specine 相对最强基线在所有测试对象上平均提升 29.60%。
Code generation refers to the automatic generation of source code based on a given programming specification, which has garnered significant attention particularly with the advancement of large language models (LLMs). However, due to the inherent complexity of real-world problems, the LLM-generated code often fails to fully align with the provided specification. While state-of-the-art agent-based techniques have been proposed to enhance LLM code generation, they overlook the critical issue of specification perception, resulting in persistent misalignment issues. Given that accurate perception of programming specifications serves as the foundation of the LLM-based code generation paradigm, ensuring specification alignment is particularly crucial. In this work, we draw on software requirements engineering to propose Specine, a novel specification alignment technique for LLM code generation. Its key idea is to identify misaligned input specifications, lift LLM-perceived specifications, and align them to enhance the code generation performance of LLMs. Our comprehensive experiments on four state-of-the-art LLMs across five challenging competitive benchmarks by comparing with ten state-of-the-art baselines, demonstrate the effectiveness of Specine. For example, Specine outperforms the most effective baseline, achieving an average improvement of 29.60% across all subjects in terms of Pass@1.