一句话：提出 k NM-LM，用检索增强与贝叶斯推断将领域知识注入语言模型，实现免微调的领域自适应代码补全。
大型语言模型在代码补全中表现优异，但缺乏领域特定知识，难以补全需要密集领域知识的代码（如库名）。已有微调方法需随项目迭代持续调整模型，且依赖参数访问。本文提出 k NM-LM，一种检索增强语言模型，将领域知识融入语言模型而无需微调。该方法利用领域内代码构建与语言模型解耦的检索数据库，再通过贝叶斯推断与语言模型结合以完成代码，可自动适应不同语言模型与领域。在项目内和场景内代码补全任务上的实验表明，k NM-LM 相比 CodeGPT 和 UnixCoder 有显著提升；对响应速度、存储占用、特定类型代码补全和 API 调用补全的分析也验证了其良好性能。该方法无需直接访问模型参数，可无缝集成到黑盒代码补全模型中作为插件进一步增强性能。
Large Language Models (LLMs) have demonstrated remarkable performance in code completion. However, due to the lack of domain-specific knowledge, they may not be optimal in completing code that requires intensive domain knowledge for example completing the library names. Although there are several works that have confirmed the effectiveness of fine-tuning techniques to adapt language models for code completion in specific domains. They are limited by the need for constant fine-tuning of the model when the project is in constant iteration. To address this limitation, in this paper, we propose $k$ NM-LM, a retrieval-augmented language model (R-LM), that integrates domain knowledge into language models without fine-tuning. Different from previous techniques, our approach is able to automatically adapt to different language models and domains. Specifically, it utilizes the in-domain code to build the retrieval-based database decoupled from LM, and then combines it with LM through Bayesian inference to complete the code. The extensive experiments on the completion of intra-project and intra-scenario have confirmed that $k$ NM-LM brings about appreciable enhancements when compared to CodeGPT and UnixCoder. A deep analysis of our tool including the responding speed, storage usage, specific type code completion, and API invocation completion has confirmed that $k$ NM-LM provides satisfactory performance, which renders it highly appropriate for domain adaptive code completion. Furthermore, our approach operates without the requirement for direct access to the language model's parameters. As a result, it can seamlessly integrate with black-box code completion models, making it easy to integrate our approach as a plugin to further enhance the performance of these models.
DOI 原文 · arXiv · PDF（开放获取） · DBLP
@inproceedings{TangGLZXHL23,
title = {Domain Adaptive Code Completion via Language Models and Decoupled Domain Databases},
author = {Ze Tang and Jidong Ge and Shangqing Liu and Tingwei Zhu and Tongtong Xu and Liguo Huang and Bin Luo},
booktitle = {ASE},
year = {2023},
doi = {10.1109/ase56229.2023.00076}
}