一句话：提出列合并打包，将稀疏卷积滤波器压缩为稠密列，提升脉动阵列利用率并保持精度。
论文针对稀疏卷积神经网络在脉动阵列上因权重稀疏导致硬件利用率低的问题，提出列合并打包方法。它将卷积滤波器矩阵的多个稀疏列合并为单个稠密列存入脉动阵列，每行仅保留幅值最大的权重并剪枝其余权重，再重训练保留权重以保持分类精度。作者基于 ASIC/FPGA 的位串行乘累加实现，联合优化阵列利用率与分类精度。实验显示非零权重密度和阵列利用率可显著提升（约 8 倍），重训练只需少量原始数据（CIFAR-10 上 10%），并在能效（3 倍）和推理延迟（12 倍）上优于先前工作。
This paper describes a novel approach of packing sparse convolutional neural networks into a denser format for efficient implementations using systolic arrays. By combining multiple sparse columns of a convolutional filter matrix into a single dense column stored in the systolic array, the utilization efficiency of the systolic array can be substantially increased (e.g., 8x) due to the increased density of nonzero weights in the resulting packed filter matrix. In combining columns, for each row, all filter weights but the one with the largest magnitude are pruned. The remaining weights are retrained to preserve high accuracy. We study the effectiveness of this joint optimization for both high utilization efficiency and classification accuracy with ASIC and FPGA designs based on efficient bit-serial implementations of multiplier-accumulators. We demonstrate that in mitigating data privacy concerns the retraining can be accomplished with only fractions of the original dataset (e.g., 10% for CIFAR-10). We present analysis and empirical evidence on the superior performance of our column combining approach against prior arts under metrics such as energy efficiency (3x) and inference latency (12x).
DOI 原文 ·
@inproceedings{paperbot4910,
title = {Packing Sparse Convolutional Neural Networks for Efficient Systolic Array Implementations},
author = {H.T. Kung and Bradley McDanel and Sai Qian Zhang},
booktitle = {ASPLOS},
year = {2019},
doi = {10.1145/3297858.3304028}
}