一句话：提出显式建模编辑的预训练目标并构建 CoditT5，用于源代码与自然语言编辑，在多个下游任务上超越生成式模型。
预训练语言模型在软件相关生成任务中表现优异，但其设计并未显式推理编辑，因而难以直接适用于编辑任务。本文提出一种新的预训练目标，显式建模源代码与自然语言注释中的编辑，并据此构建 CoditT5。该模型在大规模源代码和自然语言注释上预训练，随后在注释更新、缺陷修复和自动代码审查等下游编辑任务上微调。实验表明，CoditT5 优于标准生成式模型，说明该编辑建模范式具有通用性；进一步通过简单重排序策略将生成模型与编辑模型互补，在三个下游任务上取得 state-of-the-art 性能。
Pretrained language models have been shown to be effective in many software-related generation tasks; however, they are not well-suited for editing tasks as they are not designed to reason about edits. To address this, we propose a novel pretraining objective which explicitly models edits and use it to build CoditT5, a large language model for software-related editing tasks that is pretrained on large amounts of source code and natural language comments. We fine-tune it on various downstream editing tasks, including comment updating, bug fixing, and automated code review. By outperforming standard generation-based models, we demonstrate the generalizability of our approach and its suitability for editing tasks. We also show how a standard generation model and our edit-based model can complement one another through simple reranking strategies, with which we achieve state-of-the-art performance for the three downstream editing tasks.
@inproceedings{paperbot7328,
title = {CoditT5: Pretraining for Source Code and Natural Language Editing},
author = {Jiyang Zhang and Sheena Panthaplackel and Pengyu Nie and Junyi Jessy Li and Milos Gligoric},
booktitle = {ASE},
year = {2022},
doi = {10.1145/3551349.3556955}
}