一句话：复现并评估既有代码作者识别方法在真实项目中的效果与局限。
代码作者识别在多个研究中具有重要价值，已有方法在各自数据集上表现良好，但其真实有效性仍存疑。此前唯一的大规模研究以 Google Code Jam 程序为对象，与日常开发源文件差异较大。本文复现该研究，并将其方法应用于真实项目源文件，考察时间效应与作者文件分布等因素的影响。结果表明，与先前结论相反，在 85.48% 的训练/测试集配对中，考虑时间效应后模型准确率下降，整体平均准确率降低 0.4298；在真实文件分布下，该方法通常只能准确识别一至两名核心作者，而项目可能拥有十余名作者。该研究揭示了既有方法的局限性，并为未来改进指明方向。
Identifying code authors is important in many research topics, and various approaches have been proposed. Although these approaches achieve promising results on their datasets, their true effectiveness is still in question. To the best of our knowledge, only one large-scale study was conducted to explore the impacts of related factors (e.g., the temporal effect and the distribution of files per author). This study selected Google Code Jam programs as their subjects, but such programs are quite different from the source files that programmers write in daily development. To understand their effectiveness and challenges, we replicate their study and use their approach to analyze source files that are retrieved from real projects. The prior study claims that the temporal effect and the distribution of files per author have only minor impacts on their trained models. In the contrast, we find that in 85.48% pairs of training and testing sets, the accuracy of a trained model is less effective when the temporal effect is considered, and in total, the average accuracy decreases by 0.4298. In addition, when we use the real distribution of files as inputs, their approach can accurately identify only one or two core code authors, although a project can have more than ten authors. By revealing the limitations of the prior approach, our study sheds lights on where to make future improvements.
DOI 原文 ·
@inproceedings{paperbot7080,
title = {A study on identifying code author from real development},
author = {Siyi Gong and Hao Zhong},
booktitle = {FSE},
year = {2022},
doi = {10.1145/3540250.3560878}
}