一句话：提出 Fractal 自动调优系统与 PatternIR，将稀疏模式统一为多级稀疏分块以提升 DNN 剪枝性能。
模型剪枝可去除冗余参数并降低 DNN 计算量，但剪枝后稀疏模型的不规则访存与计算模式使性能难以提升；现有工作多采用单层结构化稀疏模式。本文从“分块后跳过计算”的统一视角重新理解稀疏模式设计，提出多级稀疏分块设计空间与自动调优系统 Fractal，以同时权衡精度与性能。Fractal 引入 PatternIR 作为高层中间表示，可表达多样多级稀疏模式，并将其翻译为底层编译器 IR，以复用稠密算子优化并完成代码生成。在 75% 稀疏度下，Fractal 相比先进稠密基线在 GPU CUDA Core 和 TensorCore 上分别平均加速 3.16× 与 2.52×，且相比现有稀疏算子库精度损失很小。
Model pruning, which eliminates redundant parameters and reduces computational complexity, emerges as a viable strategy for efficient deep neural network (DNN) deployment. Owing to the irregular memory access and computation patterns in the sparse DNN models after pruning, existing arts have suggested various structured sparse patterns to enhance sparse DNN performance. In this work, we propose a unique perspective of understanding existing sparse pattern design as computation-skipping after tiling the tensor computation into multi-level hierarchies. This unified perspective opens up a new design space of multi-level sparse tiling to maximize the sparsity benefits of DNNs, as opposed to the single-level choice in current practices. We present Fractal, an auto-tuning system for sparse patterns that identifies the optimal multi-level sparse tiling pattern. We introduce PatternIR, a novel high-level intermediate representation (IR), to express a diverse range of multi-level sparse patterns. By leveraging insights from prior dense operator optimizations, we translate PatternIR into low-level compiler IRs, facilitating further operator optimization and code generation. Our evaluations demonstrate that Fractal yields substantial speedups of up to on average 3.16× on CUDA Core, 2.52× on TensorCore of GPUs compared to the state-of-art dense baseline under 75% sparsity while upholding minimal accuracy degradation compared to prior sparse operator libraries.
@inproceedings{GuanYZLLG24,
title = {Fractal: Joint Multi-Level Sparse Pattern Tuning of Accuracy and Performance for DNN Pruning},
author = {Yue Guan and Changming Yu and Yangjie Zhou and Jingwen Leng and Chao Li and Minyi Guo},
booktitle = {ASPLOS},
year = {2024},
doi = {10.1145/3620666.3651351}
}