一句话：提出基于 token 类型序列与机器学习的代码克隆检测方法 Toma，在 BigCloneBench 上兼顾有效性与效率。
随着软件工程发展和对代码需求的增长，代码克隆日益普遍，并带来漏洞传播等风险，因此代码克隆检测愈发重要。现有方法在真实代码环境中往往表现不足：要么难以有效识别克隆，要么处理复杂克隆需要大量时间和计算资源。本文提出 Toma，一种基于 token 与机器学习的克隆检测方法。它提取 token 类型序列，使用六种相似度计算方法生成特征向量，再输入训练好的机器学习模型进行分类。为评估有效性与可扩展性，作者在广泛使用的 BigCloneBench 数据集上实验。结果表明，Toma 优于基于 token 的克隆检测器和多数基于树的检测器，并显著节省时间。
As software engineering advances and the code demand rises, the prevalence of code clones has increased. This phenomenon poses risks like vulnerability propagation, underscoring the growing importance of code clone detection techniques. While numerous code clone detection methods have been proposed, they often fall short in real-world code environments. They either struggle to identify code clones effectively or demand substantial time and computational resources to handle complex clones. This paper introduces a code clone detection method namely Toma using tokens and machine learning. Specifically, we extract token type sequences and employ six similarity calculation methods to generate feature vectors. These vectors are then input into a trained machine learning model for classification. To evaluate the effectiveness and scalability of Toma, we conduct experiments on the widely used BigCloneBench dataset. Results show that our tool outperforms token-based code clone detectors and most tree-based clone detectors, demonstrating high effectiveness and significant time savings.
@inproceedings{paperbot11435,
title = {Machine Learning is All You Need: A Simple Token-based Approach for Effective Code Clone Detection},
author = {Siyue Feng and Wenqi Suo and Yueming Wu and Deqing Zou and Yang Liu and Hai Jin},
booktitle = {ICSE},
year = {2024},
doi = {10.1145/3597503.3639114}
}