一句话：提出 DNN 编译器 Souffle，以张量表达式构建全局依赖图并做跨算子边界的优化，显著加速推理。
随着深度神经网络复杂度上升，优化其执行变得愈发困难。现有 DNN 编译器大多在算子内部做局部优化，难以跨越算子边界挖掘优化机会。本文提出开源编译器 Souffle，面向 DNN 推理实现跨算子边界的全局优化。Souffle 首先用张量表达式构造全局张量依赖图，追踪数据流与张量信息，并基于数据流分析和资源约束将计算图划分为若干子程序；在每个子程序内部，它通过保持语义等价的变换进行局部优化，搜索优化的程序调度，从而提升指令级并行度与数据复用。作者在 NVIDIA A100 GPU 上用六个代表性 DNN 模型进行评估，结果显示 Souffle 一致优于六个当前最优的 DNN 优化器，相对 TensorRT 取得最高 3.7 倍的几何平均加速，相对 TensorFlow XLA 最高达 7.8 倍。
Optimizing deep neural network (DNN) execution is important but becomes increasingly difficult as DNN complexity grows. Existing DNN compilers cannot effectively exploit optimization opportunities across operator boundaries, leaving room for improvement. To address this challenge, we present Souffle, an open-source compiler that optimizes DNN inference across operator boundaries. Souffle creates a global tensor dependency graph using tensor expressions, traces data flow and tensor information, and partitions the computation graph into subprograms based on dataflow analysis and resource constraints. Within a subprogram, Souffle performs local optimization via semantic-preserving transformations, finds an optimized program schedule, and improves instruction-level parallelism and data reuse. We evaluated Souffle using six representative DNN models on an NVIDIA A100 GPU. Experimental results show that Souffle consistently outperforms six state-of-the-art DNN optimizers by delivering a geometric mean speedup of up to 3.7× over TensorRT and 7.8× over Tensorflow XLA.
@inproceedings{XiaZSWWYFC24,
title = {Optimizing Deep Learning Inference via Global Analysis and Tensor Expressions},
author = {Chunwei Xia and Jiacheng Zhao and Qianqi Sun and Zheng Wang and Yuan Wen and Teng Yu and Xiaobing Feng and Huimin Cui},
booktitle = {ASPLOS},
year = {2024},
doi = {10.1145/3617232.3624858}
}