一句话：提出首个动态技术 GRIST，利用梯度反向传播自动生成小输入以暴露深度学习程序中的数值缺陷。
深度学习程序以数值计算为主，数值缺陷是其主要缺陷之一。特殊输入可能导致 log() 等内部数学操作出现非法参数值，进而产生 NaN 或 INF 并传播，最终引发崩溃或无效输出。本文提出首个动态技术 GRIST，它借助深度学习基础设施内置的梯度计算功能，通过梯度反向传播自动生成能够暴露数值缺陷的小输入。作者在 63 个真实深度学习程序上评估，GRIST 检测到 78 个缺陷，其中 56 个为未知缺陷；提交至相应 issue 仓库后，8 个被确认、3 个被修复。相比用所提供的输入运行原程序，GRIST 可节省 8.79 倍执行时间。与静态技术 DEBAR 相比，DEBAR 产生 12 个假阳性并漏报 31 个真实缺陷（其中 30 个可被 GRIST 发现），而 GRIST 仅漏报一个已知缺陷且无假阳性，证明了其有效性。
Numerical computation is dominant in deep learning (DL) programs. Consequently, numerical bugs are one of the most prominent kinds of defects in DL programs. Numerical bugs can lead to exceptional values such as NaN (Not-a-Number) and INF (Infinite), which can be propagated and eventually cause crashes or invalid outputs. They occur when special inputs cause invalid parameter values at internal mathematical operations such as log(). In this paper, we propose the first dynamic technique, called GRIST, which automatically generates a small input that can expose numerical bugs in DL programs. GRIST piggy-backs on the built-in gradient computation functionalities of DL infrastructures. Our evaluation on 63 real-world DL programs shows that GRIST detects 78 bugs including 56 unknown bugs. By submitting them to the corresponding issue repositories, eight bugs have been confirmed and three bugs have been fixed. Moreover, GRIST can save 8.79X execution time to expose numerical bugs compared to running original programs with its provided inputs. Compared to the state-of-the-art technique DEBAR (which is a static technique), DEBAR produces 12 false positives and misses 31 true bugs (of which 30 bugs can be found by GRIST), while GRIST only misses one known bug in those programs and no false positive. The results demonstrate the effectiveness of GRIST.
DOI 原文 ·
@inproceedings{paperbot6473,
title = {Exposing numerical bugs in deep learning via gradient back-propagation},
author = {Ming Yan and Junjie Chen and Xiangyu Zhang and Lin Tan and Gan Wang and Zan Wang},
booktitle = {FSE},
year = {2021},
doi = {10.1145/3468264.3468612}
}