一句话：提出轻量算子规格生成多样有效 DNN 模型、梯度搜索规避浮点异常并结合差分测试的深度学习编译器模糊测试方法，发现 72 个新 bug。
深度学习编译器（如 TVM、TensorRT）被广泛用于优化 DNN 模型，但其复杂实现中的 bug 会导致编译后模型语义与原模型不一致，从而破坏下游应用的正确性。由于编译器内部变换逻辑复杂，传统测试难以有效暴露此类缺陷。本文提出一种新的模糊测试方法：其一，借助轻量的算子规格生成多样而又语义有效的 DNN 测试模型，以覆盖编译器大量的变换逻辑；其二，采用基于梯度的搜索为模型寻找合适的输入，避免执行过程中出现浮点异常值，从而减少漏报与误报；其三，利用差分测试在多个编译器/后端之间比对结果以识别 bug。作者将该方法实现为 NNSmith，并已为 TVM、TensorRT、ONNXRuntime 和 PyTorch 发现 72 个新 bug，其中 58 个已被确认、51 个已被相应维护者修复。
Deep-learning (DL) compilers such as TVM and TensorRT are increasingly being used to optimize deep neural network (DNN) models to meet performance, resource utilization and other requirements. Bugs in these compilers can result in models whose semantics differ from the original ones, producing incorrect results that corrupt the correctness of downstream applications. However, finding bugs in these compilers is challenging due to their complexity. In this work, we propose a new fuzz testing approach for finding bugs in deep-learning compilers. Our core approach consists of (i) generating diverse yet valid DNN test models that can exercise a large part of the compiler's transformation logic using light-weight operator specifications; (ii) performing gradient-based search to find model inputs that avoid any floating-point exceptional values during model execution, reducing the chance of missed bugs or false alarms; and (iii) using differential testing to identify bugs. We implemented this approach in NNSmith which has found 72 new bugs for TVM, TensorRT, ONNXRuntime, and PyTorch to date. Of these 58 have been confirmed and 51 have been fixed by their respective project maintainers.
@inproceedings{paperbot10025,
title = {NNSmith: Generating Diverse and Valid Test Cases for Deep Learning Compilers},
author = {Jiawei Liu and Jinkun Lin and Fabian Ruffy and Cheng Tan and Jinyang Li and Aurojit Panda and Lingming Zhang},
booktitle = {ASPLOS},
year = {2023},
doi = {10.1145/3575693.3575707}
}