一句话：让多个量化分辨率复用同一组量化项，一次训练即可在推理时动态切换分辨率，并配套多分辨率 MAC 硬件。
针对深度神经网络低比特（如 4-bit）均匀量化通常只支持单一分辨率、推理时无法按计算预算灵活伸缩的问题，本文提出一种新的训练方法：让多个分辨率共享同一组量化项，即权重与数据中相同的非零比特集合，从而用一次训练得到可在多分辨率下复用的模型，并在推理阶段动态选择分辨率级别。作者在多种应用上验证了该方法，包括 ImageNet 上的多个 CNN、Wikitext-2 上的 LSTM 以及 COCO 上的 YOLO-v5。结果显示，多分辨率训练得到的模型可支持多达 10 种分辨率，相较逐分辨率单独训练仅有轻微性能下降（例如不超过 1%）。最后，作者在 FPGA 上实现了多分辨率乘累加单元（mMAC），与常规 MAC 设计进行比较，表明 mMAC 在不同计算预算下为成本、效率与延迟的权衡提供了更广的选择空间。
Low-resolution uniform quantization (e.g., 4-bit bitwidth) for both Deep Neural Network (DNN) weights and data has emerged as an important technique for efficient inference. Departing from conventional quantization, we describe a novel training approach to support inference at multiple resolutions by reusing a single set of quantization terms (the same set of nonzero bits in values). The proposed approach streamlines the training and supports dynamic selection of resolution levels during inference. We evaluate the method on a diverse range of applications including multiple CNNs on ImageNet, an LSTM on Wikitext-2, and YOLO-v5 on COCO. We show that models resulting from our multi-resolution training can support up to 10 resolutions with only a moderate performance reduction (e.g., ≤ 1%) compared to training them individually. Lastly, using an FPGA, we compare our multi-resolution multiplier-accumulator (mMAC) against other conventional MAC designs and evaluate the inference performance. We show that the mMAC design broadens the choices in trading off cost, efficiency, and latency across a range of computational budgets.
DOI 原文 ·
@inproceedings{paperbot6093,
title = {Training for multi-resolution inference using reusable quantization terms},
author = {Sai Qian Zhang and Bradley McDanel and H. T. Kung and Xin Dong},
booktitle = {ASPLOS},
year = {2021},
doi = {10.1145/3445814.3446741}
}