一句话：提出 GrammarT5，一种集成语法的预训练编码器-解码器代码模型，以更好捕获代码结构。
当前多数代码预训练模型仍将代码表示为 token 序列，可能无法充分捕获代码的底层结构。为此，论文提出 GrammarT5，一种将语法信息集成到预训练编码器-解码器神经网络（基于 T5）中的模型，使模型在预训练阶段能够利用代码的语法结构。该方法旨在提升模型对代码结构的建模能力，并可应用于代码摘要、代码补全、代码翻译和缺陷检测等多种代码相关任务。摘要未给出具体实验结果，但指出预训练代码模型在这些任务上已展现出良好性能，而 GrammarT5 通过语法集成进一步改进对代码结构的表示。
Pretrained models for code have exhibited promising performance across various code-related tasks, such as code summarization, code completion, code translation, and bug detection. However, despite their success, the majority of current models still represent code as a token sequence, which may not adequately capture the essence of the underlying code structure.
@inproceedings{paperbot11378,
title = {GrammarT5: Grammar-Integrated Pretrained Encoder-Decoder Neural Model for Code},
author = {Qihao Zhu and Qingyuan Liang and Zeyu Sun and Yingfei Xiong and Lu Zhang and Shengyu Cheng},
booktitle = {ICSE},
year = {2024},
doi = {10.1145/3597503.3639125}
}