一句话：提出 AST-Trans，利用祖先-后代与兄弟关系施加树结构注意力编码 AST，实现更准更快的代码摘要。
代码摘要旨在为源代码生成简短的自然语言描述，当前主流方法采用基于 Transformer 的编码器-解码器架构，并借助抽象语法树（AST）编码结构信息。然而 AST 比对应源码长得多，已有工作忽略其长度约束，直接将整棵线性化 AST 送入编码器。作者认为这种做法既难以从过长序列中提取真正有用的依赖关系，也因每个节点都要与所有其他节点做自注意力而带来显著计算开销。为此本文提出 AST-Trans，利用 AST 中两类节点关系——祖先-后代关系与兄弟关系——施加树结构注意力，据此动态为相关节点分配权重并排除无关节点，同时给出支持快速并行计算的高效实现。在两个代码摘要数据集上，AST-Trans 显著优于现有最优方法，且效率为标准 Transformer 的数倍。
Code summarization aims to generate brief natural language descriptions for source codes. The state-of-the-art approaches follow a transformer-based encoder-decoder architecture. As the source code is highly structured and follows strict grammars, its Abstract Syntax Tree (AST) is widely used for encoding structural information. However, ASTs are much longer than the corresponding source code. Existing approaches ignore the size constraint and simply feed the whole linearized AST into the encoders. We argue that such a simple process makes it difficult to extract the truly useful dependency relations from the overlong input sequence. It also incurs significant computational overhead since each node needs to apply self-attention to all other nodes in the AST. To encode the AST more effectively and efficiently, we propose AST-Trans in this paper which exploits two types of node relationships in the AST: ancestor-descendant and sibling relationships. It applies the tree-structured attention to dynamically allocate weights for relevant nodes and exclude irrelevant nodes based on these two relationships. We further propose an efficient implementation to support fast parallel computation for tree-structure attention. On the two code summarization datasets, experimental results show that AST-Trans significantly outperforms the state-of-the-arts while being times more efficient than standard transformers1.
DOI 原文 ·
@inproceedings{paperbot6917,
title = {AST-trans},
author = {Ze Tang and Xiaoyu Shen and Chuanyi Li and Jidong Ge and Liguo Huang and Zhelin Zhu and Bin Luo},
booktitle = {ICSE},
year = {2022},
doi = {10.1145/3510003.3510224}
}