一句话：实证表明大语言模型通过上下文学习即可生成满足多种开发者意图的代码注释，并显著超越有监督方法。
代码注释生成旨在为代码片段生成自然语言描述以辅助程序理解，但现有方法通常只能生成单一注释，难以满足开发者对功能、用法等多角度信息的需求。本文实证研究利用大语言模型（LLM）生成多意图注释的可行性，其直觉是 LLM 预训练中已建立代码与注释的语义关联，且真实注释包含多样开发者意图。作者采用上下文学习范式，通过设计充足提示（如提供十多个示例）与定制化提示构造及结果重排序策略，在两个大规模数据集上评估。结果显示，LLM 在多意图注释生成上显著优于最先进的有监督学习方法，且提示构造与后处理重排序均能进一步提升性能，为后续研究提供了方向。
Code comment generation aims at generating natural language descriptions for a code snippet to facilitate developers' program comprehension activities. Despite being studied for a long time, a bottleneck for existing approaches is that given a code snippet, they can only generate one comment while developers usually need to know information from diverse perspectives such as what is the functionality of this code snippet and how to use it. To tackle this limitation, this study empirically investigates the feasibility of utilizing large language models (LLMs) to generate comments that can fulfill developers' diverse intents. Our intuition is based on the facts that (1) the code and its pairwise comment are used during the pre-training process of LLMs to build the semantic connection between the natural language and programming language, and (2) comments in the real-world projects, which are collected for the pre-training, usually contain different developers' intents. We thus postulate that the LLMs can already understand the code from different perspectives after the pre-training. Indeed, experiments on two large-scale datasets demonstrate the rationale of our insights: by adopting the in-context learning paradigm and giving adequate prompts to the LLM (e.g., providing it with ten or more examples), the LLM can significantly outperform a state-of-the-art supervised learning approach on generating comments with multiple intents. Results also show that customized strategies for constructing the prompts and post-processing strategies for reranking the results can both boost the LLM's performances, which shed light on future research directions for using LLMs to achieve comment generation.
DOI 原文 ·
@inproceedings{paperbot11466,
title = {Large Language Models are Few-Shot Summarizers: Multi-Intent Comment Generation via In-Context Learning},
author = {Mingyang Geng and Shangwen Wang and Dezun Dong and Haotian Wang and Ge Li and Zhi Jin and Xiaoguang Mao and Xiangke Liao},
booktitle = {ICSE},
year = {2024},
doi = {10.1145/3597503.3608134}
}