一句话：提出 OMEGA，用 4-bit 量化的 8B 开源 LLM 结合上下文细化生成提交信息，在从业者偏好上超越 GPT-4。
提交信息以自然语言描述代码改动，对软件维护与演化至关重要。近期 LLM 被用于自动生成高质量提交信息，如基于 GPT-4 的 OMG，但专有模型带来隐私与可持续性隐患，阻碍工业落地。本文探究开源 LLM 能否生成与 OMG 相当的提交信息。实验显示开源 LLM 可达到可比质量；作者进一步通过一系列上下文细化提出 OMEGA，使用 4-bit 量化的 8B 开源模型。评估表明 OMEGA 生成的提交信息达到当前最佳水平，并在从业者偏好上超过 GPT-4。
Commit messages provide descriptions of the modifications made in a commit using natural language, making them crucial for software maintenance and evolution. Recent developments in Large Language Models (LLMs) have led to their use in generating high-quality commit messages, such as the Omniscient Message Generator (OMG). This method employs GPT-4 to produce state-of-the-art commit messages. However, the use of proprietary LLMs like GPT-4 in coding tasks raises privacy and sustainability concerns, which may hinder their industrial adoption. Considering that open-source LLMs have achieved competitive performance in developer tasks such as compiler validation, this study investigates whether they can be used to generate commit messages that are comparable with OMG. Our experiments show that an open-source LLM can generate commit messages comparable to those produced by OMG. In addition, through a series of contextual refinements, we propose OMEGA, a commit message generation approach that uses a 4-bit quantized 8B open-source LLM. OMEGA produces state-of-the-art commit messages, surpassing the performance of GPT-4 in practitioners' preference.
DOI 原文 ·
@inproceedings{paperbot8881,
title = {Context Conquers Parameters: Outperforming Proprietary Llm in Commit Message Generation},
author = {Aaron Imani and Iftekhar Ahmed and Mohammad Moshirpour},
booktitle = {ICSE},
year = {2025},
doi = {10.1109/icse55347.2025.00048}
}