一句话：提出 Tzer，通过联合 IR 与 pass 变异和覆盖率反馈对 TVM 张量编译器进行模糊测试，发现 49 个新缺陷。
随着深度学习系统广泛部署，其底层张量编译器（如 TVM）的可靠性至关重要，但相关测试工作仍然有限。本文针对张量编译器测试问题提出 Tzer，一种面向 TVM 的实用模糊测试技术。由于高层 IR 变异空间有限，Tzer 聚焦于低层中间表示（IR）的变异，结合通用变异器与张量编译器专用变异器，并以覆盖率反馈引导实现多样且进化的 IR 变异；同时，鉴于张量编译器提供多种 pass 进行 IR 优化，Tzer 将 pass 变异与 IR 变异协同进行以提升模糊测试效果。实验表明，Tzer 在张量编译器测试上显著优于现有模糊测试技术，覆盖率提升 75%，有价值测试数量比第二名多 50%，并通过消融实验验证了各组件作用。目前 Tzer 已为 TVM 发现 49 个未知缺陷，其中 37 个被确认，25 个被修复（PR 合并）。
In the past decade, Deep Learning (DL) systems have been widely deployed in various application domains to facilitate our daily life, e.g., natural language processing, healthcare, activity recognition, and autonomous driving. Meanwhile, it is extremely challenging to ensure the correctness of DL systems (e.g., due to their intrinsic nondeterminism), and bugs in DL systems can cause serious consequences and may even threaten human lives. In the literature, researchers have explored various techniques to test, analyze, and verify DL models, since their quality directly affects the corresponding system behaviors. Recently, researchers have also proposed novel techniques for testing the underlying operator-level DL libraries (such as TensorFlow and PyTorch), which provide general binary implementations for each high-level DL operator and are the foundation for running DL models on different hardware platforms. However, there is still limited work targeting the reliability of the emerging tensor compilers (also known as DL compilers), which aim to automatically compile high-level tensor computation graphs directly into high-performance binaries for better efficiency, portability, and scalability than traditional operator-level libraries. Therefore, in this paper, we target the important problem of tensor compiler testing, and have proposed Tzer, a practical fuzzing technique for the widely used TVM tensor compiler. Tzer focuses on mutating the low-level Intermediate Representation (IR) for TVM due to the limited mutation space for the high-level IR. More specifically, Tzer leverages both general-purpose and tensor-compiler-specific mutators guided by coverage feedback for diverse and evolutionary IR mutation; furthermore, since tensor compilers provide various passes (i.e., transformations) for IR optimization, Tzer also performs pass mutation in tandem with IR mutation for more effective fuzzing. Our experimental results show that Tzer substantially outperforms existing fuzzing techniques on tensor compiler testing, with 75% higher coverage and 50% more valuable tests than the 2nd-best technique. Also, different components of Tzer have been validated via ablation study. To date, Tzer has detected 49 previously unknown bugs for TVM, with 37 bugs confirmed and 25 bugs fixed (PR merged).
@article{paperbot1530,
title = {Coverage-guided tensor compiler fuzzing with joint IR-pass mutation},
author = {Jiawei Liu and Yuxiang Wei and Sen Yang and Yinlin Deng and Lingming Zhang},
journal = {Proceedings of the ACM on Programming Languages},
volume = {6},
number = {OOPSLA1},
year = {2022},
doi = {10.1145/3527317}
}