一句话：提出面向代码生成的推理加速方法 FastCoder，用多源检索草稿与验证实现最高约 2.5 倍加速且不损质量。
针对仓库级代码生成延迟敏感、现有 LLM 推理加速方法忽视代码语法语义导致效果有限的问题，提出 FastCoder。它采用 draft-verification 范式，构建包含通用与项目特定知识的多源 datastore 以检索高质量草稿，并通过控制检索时机、并行检索及上下文与 LLM 偏好感知缓存降低开销。实验表明，FastCoder 在仓库级和独立代码生成任务上分别取得最高 2.53× 和 2.54× 的加速，较最先进推理加速方法最高提升 88%；与正确性导向的代码生成方法集成后加速仍可超过 2.6×，且不损害输出质量。
Code generation is a latency-sensitive task that demands high timeliness. However, with the growing interest and inherent difficulty in repository-level code generation, most existing code generation studies focus on improving the correctness of generated code while overlooking the inference efficiency, which is substantially affected by the overhead during LLM generation. Although there has been work on accelerating LLM inference, these approaches are not tailored to the specific characteristics of code generation; instead, they treat code the same as natural language sequences and ignore its unique syntax and semantic characteristics, which are also crucial for improving efficiency. Consequently, these approaches exhibit limited effectiveness in code generation tasks, particularly for repository-level scenarios with considerable complexity and difficulty. To alleviate this issue, following draft-verification paradigm, we propose FastCoder, a simple yet highly efficient inference acceleration approach specifically designed for code generation, without compromising the quality of the output. FastCoder constructs a multi-source datastore, providing access to both general and project-specific knowledge, facilitating the retrieval of high-quality draft sequences. Moreover, FastCoder reduces the retrieval cost by controlling retrieval timing, and enhances efficiency through parallel retrieval and a context- and LLM preference-aware cache. Experimental results show that FastCoder can reach up to 2.53× and 2.54× speedup compared to autoregressive decoding in repository-level and standalone code generation tasks, respectively, outperforming state-of-the-art inference acceleration approaches by up to 88%. FastCoder can also be integrated with existing correctness-focused code generation approaches to accelerate the LLM generation process, and reach a speedup exceeding 2.6×.
DOI 原文 ·
@inproceedings{paperbot9415,
title = {FastCoder: Accelerating Repository-level Code Generation via Efficient Retrieval and Verification},
author = {Qianhui Zhao and Li Zhang and Fang Liu and Xiaoli Lian and Qiaoyuanhe Meng and Ziqian Jiao and Zetong Zhou and Jia Li and Lin Shi},
booktitle = {ASE},
year = {2025},
doi = {10.1109/ase63991.2025.00190}
}