一句话：揭示神经代码克隆检测缺乏跨功能泛化能力，并给出评估方法、影响因素分析与人在环适配机制。
论文指出当前神经代码克隆检测评估方法存在一个关键疏漏：未考察跨功能泛化，即检测训练中从未出现过的功能所对应的语义克隆。作者首先提出一套能系统度量跨功能泛化性的评估方法，并据此开展实证研究，发现现有神经克隆检测器在新功能上的泛化能力远低于预期。随后论文分析影响泛化性的三个关键因素：训练数据多样性、词表（out-of-vocabulary）问题与局部性，结果表明缓解未登录词问题、提升训练数据多样性可显著降低未见功能上的性能损失。最后作者提出一种人在环机制，将迁移学习与主动学习结合，帮助检测器适配包含大量未见功能的新代码仓库，实验显示该机制可减少约 88% 的人工标注量。代码与数据已公开。
Many recently proposed code clone detectors exploit neural networks to capture latent semantics of source code, thus achieving impressive results for detecting semantic clones. These neural clone detectors rely on the availability of large amounts of labeled training data. We identify a key oversight in the current evaluation methodology for neural clone detection: cross-functionality generalization (i.e., detecting semantic clones of which the functionalities are unseen in training). Specifically, we focus on this question: do neural clone detectors truly learn the ability to detect semantic clones, or they just learn how to model specific functionalities in training data while cannot generalize to realistic unseen functionalitiesƒ This paper investigates how the generalizability can be evaluated and improved.Our contributions are 3-folds: (1) We propose an evaluation methodology that can systematically measure the cross-functionality generalizability of neural clone detection. Based on this evaluation methodology, an empirical study is conducted and the results indicate that current neural clone detectors cannot generalize well as expected. (2) We conduct empirical analysis to understand key factors that can impact the generalizability. We investigate 3 factors: training data diversity, vocabulary, and locality. Results show that the performance loss on unseen functionalities can be reduced through addressing the out-of-vocabulary problem and increasing training data diversity. (3) We propose a human-in-the-loop mechanism that help adapt neural clone detectors to new code repositories containing lots of unseen functionalities. It improves annotation efficiency with the combination of transfer learning and active learning. Experimental results show that it reduces the amount of annotations by about 88%. Our code and data are publicly available1.
DOI 原文 ·
@inproceedings{paperbot6509,
title = {Can Neural Clone Detection Generalize to Unseen Functionalitiesƒ},
author = {Chenyao Liu and Zeqi Lin and Jian-Guang Lou and Lijie Wen and Dongmei Zhang},
booktitle = {ASE},
year = {2021},
doi = {10.1109/ase51524.2021.9678907}
}