一句话：独立复现 DeepBugs 的换参 bug 检测，准确率与原报告相差 2% 以内。
本文对 DeepBugs 这一基于命名的 bug 检测算法进行复现。原工作在三种场景下评估：参数顺序颠倒、二元运算符错误以及二元运算符操作数错误。作者针对参数顺序颠倒（swapped-argument）场景，独立重新实现了主要组件，包括训练集生成、token 向量化，以及神经网络的数据流水线、模型与损失函数。复现沿用与原工作相同的数据集和测试流程，报告了与原结果相当的性能：准确率与 Pradel 和 Sen 报告的数值相差在 2% 以内，表明该结果在组件独立实现的情况下具有可复现性。
We replicated the main result of DeepBugs, a bug detection algorithm for name-based bugs. The original authors evaluated it in three contexts: swapped-argument bugs, wrong binary operator,and wrong binary operator operands. We followed the algorithm and replicated the results for swapped-argument bugs. Our replication used independent implementations of the major components: training set generation, token vectorization, and neural network data pipeline, model, and loss function. Using the same dataset and the same testing process, we report comparable performance: within 2% of the accuracy reported by Pradel and Sen.
@inproceedings{paperbot6410,
title = {A replication of ‘DeepBugs: a learning approach to name-based bug detection’},
author = {Jordan Winkler and Abhimanyu Agarwal and Caleb Tung and Dario Rios Ugalde and Young Jin Jung and James C. Davis},
booktitle = {FSE},
year = {2021},
doi = {10.1145/3468264.3477221}
}