一句话：提出基于历史上下文的提交信息补全任务，构建大规模数据集并评估现有模型与GPT-3.5-turbo的表现。
提交信息对软件开发至关重要，但撰写高质量信息耗时且多数信息缺乏关键内容，现有提交信息生成（CMG）研究尚未被广泛采用。作者主张从生成转向补全，并利用先前提交历史作为额外上下文，以提升质量与个性化。为此，作者构建并公开 CommitChronicle 数据集，包含 20 种编程语言的 1070 万个提交。基于该数据集，作者评估了最先进 CMG 模型与 GPT-3.5-turbo 在补全设置及历史上下文下的表现。结果显示，某些场景下补全优于生成；GPT-3.5-turbo 总体较差但对长而详细的消息有潜力；历史信息能提升 CMG 模型的生成性能以及 GPT-3.5-turbo 在生成和补全中的性能。
Commit messages are crucial to software development, allowing developers to track changes and collaborate effectively. Despite their utility, most commit messages lack important information since writing high-quality commit messages is tedious and time-consuming. The active research on commit message generation (CMG) has not yet led to wide adoption in practice. We argue that if we could shift the focus from commit message generation to commit message completion and use previous commit history as additional context, we could significantly improve the quality and the personal nature of the resulting commit messages. In this paper, we propose and evaluate both of these novel ideas. Since the existing datasets lack historical data, we collect and share a novel dataset called CommitChronicle, containing 10.7M commits across 20 programming languages. We use this dataset to evaluate the completion setting and the usefulness of the historical context for state-of-the-art CMG models and GPT-3.5-turbo. Our results show that in some contexts, commit message completion shows better results than generation, and that while in general GPT-3.5-turbo performs worse, it shows potential for long and detailed messages. As for the history, the results show that historical information improves the performance of CMG models in the generation task, and the performance of GPT-3.5-turbo in both generation and completion.
DOI 原文 · arXiv · PDF（开放获取） · DBLP
@inproceedings{EliseevaSBGDB23,
title = {From Commit Message Generation to History-Aware Commit Message Completion},
author = {Aleksandra Eliseeva and Yaroslav Sokolov and Egor Bogomolov and Yaroslav Golubev and Danny Dig and Timofey Bryksin},
booktitle = {ASE},
year = {2023},
doi = {10.1109/ase56229.2023.00078}
}