一句话：通过分析 GitHub、Stack Overflow 与访谈，构建并验证深度学习系统真实故障的大规模分类体系。
深度学习在安全关键领域的应用日益广泛，分析其系统中真实故障至关重要。本文构建了深度学习系统真实故障的大规模分类体系。作者人工分析了来自 TensorFlow、Keras 和 PyTorch 项目的 GitHub 提交与 issue 以及相关 Stack Overflow 帖子的 1059 个制品，并对 20 名研究者和从业者进行结构化访谈以补充故障类型。最终分类体系经 21 名开发者参与调查验证，其中 15 个故障类别中有 13 个被至少 50% 的受访者确认经历过。
The growing application of deep neural networks in safety-critical domains makes the analysis of faults that occur in such systems of enormous importance. In this paper we introduce a large taxonomy of faults in deep learning (DL) systems. We have manually analysed 1059 artefacts gathered from GitHub commits and issues of projects that use the most popular DL frameworks (TensorFlow, Keras and PyTorch) and from related Stack Overflow posts. Structured interviews with 20 researchers and practitioners describing the problems they have encountered in their experience have enriched our taxonomy with a variety of additional faults that did not emerge from the other two sources. Our final taxonomy was validated with a survey involving an additional set of 21 developers, confirming that almost all fault categories (13/15) were experienced by at least 50% of the survey participants.
DOI 原文 ·
@inproceedings{paperbot5609,
title = {Taxonomy of real faults in deep learning systems},
author = {Nargiz Humbatova and Gunel Jahangirova and Gabriele Bavota and Vincenzo Riccio and Andrea Stocco and Paolo Tonella},
booktitle = {ICSE},
year = {2020},
doi = {10.1145/3377811.3380395}
}