一句话：提出 Bit-Tactical，通过静态调度中间件与轻量稀疏混洗硬件前端协同利用权重与激活的位稀疏性来加速 DNN 推理。
DNN 推理中权重和激活稀疏可提升硬件性能与能效，但充分利用稀疏需重新调度并映射执行流，将非零权重/激活对送达乘法单元；任意重调度会给硬件带来动态路由匹配负担和显著能效开销。Bit-Tactical（TCL）将利用权重稀疏的职责分摊给静态调度中间件与协同设计硬件前端；前端采用轻量稀疏混洗网络，每个激活输入仅需两个 2 至 8 输入多路复用器。其两个后端设计针对激活的位稀疏而非值稀疏，避免处理动态稀疏的整值激活流并暴露更多无效工作。在多种神经网络上，TCL 优于针对权重/激活稀疏、激活动态精度或位级稀疏的现有最优加速器。
Weight and activation sparsity can be leveraged in hardware to boost the performance and energy efficiency of Deep Neural Networks during inference. Fully capitalizing on sparsity requires re-scheduling and mapping the execution stream to deliver non-zero weight/activation pairs to multiplier units for maximal utilization and reuse. However, permitting arbitrary value re-scheduling in memory space and in time places a considerable burden on hardware to perform dynamic at-runtime routing and matching of values, and incurs significant energy inefficiencies. Bit-Tactical (TCL) is a neural network accelerator where the responsibility for exploiting weight sparsity is shared between a novel static scheduling middleware, and a co-designed hardware front-end with a lightweight sparse shuffling network comprising two (2- to 8-input) multiplexers per activation input. We empirically motivate two back-end designs chosen to target bit-sparsity in activations, rather than value-sparsity, with two benefits: a) we avoid handling the dynamically sparse whole-value activation stream, and b) we uncover more ineffectual work. TCL outperforms other state-of-the-art accelerators that target sparsity for weights and activations, the dynamic precision requirements of activations, or their bit-level sparsity for a variety of neural networks.
DOI 原文 ·
@inproceedings{paperbot4866,
title = {Bit-Tactical},
author = {Alberto Delmas Lascorz and Patrick Judd and Dylan Malone Stuart and Zissis Poulos and Mostafa Mahmoud and Sayeh Sharify and Milos Nikolic and Kevin Siu and Andreas Moshovos},
booktitle = {ASPLOS},
year = {2019},
doi = {10.1145/3297858.3304041}
}