paperbot · PL 论文追踪

全部论文 待读 收藏

P2 CRADLE: Cross-Backend Validation to Detect and Localize Bugs in Deep Learning Libraries

CRADLE：跨后端验证以检测与定位深度学习库中的缺陷
ICSE 2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE)2019匹配度 3/10 · 论文关注深度学习库（TensorFlow/CNTK/Theano）中的缺陷检测与定位，属于 ML 系统可靠性方向，与用户关注的'机器学习系统'有交集；但其核心方法是运行时跨实现差分测试与异常传播定位，既非静态分析应用于 ML/Agentic 程序，也不涉及 LLM、浮点程序分析或贝叶斯程序分析，与画像中三条主线都无直接对接。引用 223 inbox
Hung Viet Pham, Thibaud Lutellier, Weizhen Qi, Lin Tan
测试、调试与修复机器学习与程序语言
crossref: 2026-09-12dblp: 2026-09-12

AI 速览

一句话：提出 CRADLE，用跨实现不一致性检查检测深度学习库缺陷，并通过异常传播分析定位故障函数。

深度学习系统可靠性要求高，但测试困难：现有工作主要测试模型而非库实现，核心难点在于难以获知给定输入下库应有的输出。本文利用不同深度学习库对同一算法的多种实现，提出 CRADLE：先做跨实现不一致性检查以检测库中缺陷，再借助异常传播追踪与分析，定位造成不一致的故障函数。作者在 TensorFlow、CNTK、Theano 三个库、11 个数据集（含 ImageNet、MNIST、KGS Go）和 30 个预训练模型上评估，共检测出 12 个缺陷和 104 个唯一不一致，并为全部 104 个不一致都标出了与其成因相关的函数。

问题
深度学习系统对可靠性要求高，但现有测试工作聚焦于模型而非库实现；由于无法预知给定输入下库的期望输出，测试深度学习库十分困难。
方法
利用不同深度学习库中同一算法的多种实现，进行跨实现不一致性检查以检测缺陷，并通过异常传播追踪与分析定位导致不一致的故障函数。
评估
在 TensorFlow、CNTK、Theano 三个库、11 个数据集（包括 ImageNet、MNIST 和 KGS Go）与 30 个预训练模型上评估，检测出 12 个缺陷和 104 个唯一不一致，并为全部 104 个不一致定位出相关函数。
原文摘要（Abstract）

Deep learning (DL) systems are widely used in domains including aircraft collision avoidance systems, Alzheimer's disease diagnosis, and autonomous driving cars. Despite the requirement for high reliability, DL systems are difficult to test. Existing DL testing work focuses on testing the DL models, not the implementations (e.g., DL software libraries) of the models. One key challenge of testing DL libraries is the difficulty of knowing the expected output of DL libraries given an input instance. Fortunately, there are multiple implementations of the same DL algorithms in different DL libraries. Thus, we propose CRADLE, a new approach that focuses on finding and localizing bugs in DL software libraries. CRADLE (1) performs cross-implementation inconsistency checking to detect bugs in DL libraries, and (2) leverages anomaly propagation tracking and analysis to localize faulty functions in DL libraries that cause the bugs. We evaluate CRADLE on three libraries (TensorFlow, CNTK, and Theano), 11 datasets (including ImageNet, MNIST, and KGS Go game), and 30 pre-trained models. CRADLE detects 12 bugs and 104 unique inconsistencies, and highlights functions relevant to the causes of inconsistencies for all 104 unique inconsistencies.

链接与引用

DOI 原文 · DBLP

BibTeX 
@inproceedings{PhamLQT19,
  title = {CRADLE: Cross-Backend Validation to Detect and Localize Bugs in Deep Learning Libraries},
  author = {Hung Viet Pham and Thibaud Lutellier and Weizhen Qi and Lin Tan},
  booktitle = {ICSE},
  year = {2019},
  doi = {10.1109/icse.2019.00107}
}