一句话：通过将线性层 GEMM 替换为查找表并自动调优映射，扩展商用 DRAM-PIM 对深度学习的适用性。
DRAM-PIM 已实现商用，但其算力有限，主要适合 element-wise 和 GEMV 算子，而这些算子在多数 DNN 负载中耗时占比较小，重算力算子仍需主机处理。本文提出 PIM-DL 框架，用查找表替代线性层中的 GEMM，形成 LUT-NN，以减少 DNN 推理中的乘法并适配 DRAM-PIM。为准确转换模型并优化推理服务，作者提出增强 LUT-NN 标定算法 eLUT-NN，并设计 Auto-Tuner 优化不同 DRAM-PIM 平台上的映射参数。在现成 UPMEM PIM-DIMM 与模拟 HBM-PIM/AiM 平台上评估多种当代 DNN 负载。相比 DRAM-PIM 上的 GEMM 推理，PIM-DL 取得 22.6×~37.1× 加速；相比 CPU/GPU 推理，最高取得 3.54×/1.20× 加速。
DRAM-based processing-in-memory (DRAM-PIM) has gained commercial prominence in recent years. However, their integration for deep learning acceleration poses inherent challenges. Existing DRAM-PIMs are limited in computational capabilities, primarily applicable for element-wise and GEMV operators. Unfortunately, these operators contribute only a small portion of the execution time in most DNN workloads. Current systems still necessitate powerful hosts to handle a significant portion of compute-heavy operators. To expand the applicability of commodity DRAM-PIMs in accelerating deep learning, we introduce a novel PIM-DL framework. The philosophy behind PIM-DL is to replace the compute-heavy GEMM operations in linear layers with Lookup-Tables (LUTs). Such LUT-based neural networks (LUT-NNs) substantially reduce multiplications in DNN inference, rendering them suitable for efficient execution on DRAM-PIMs. To accurately convert DNNs into LUT-NNs and achieve optimal inference serving performance, we first introduce an enhanced LUT-NN (eLUT-NN) algorithm for model calibration, then we propose an Auto-Tuner capable of optimizing the mapping parameters on diverse DRAM-PIM platforms. We evaluate PIM-DL on off-the-shelf UPMEM PIM-DIMM products and simulated HBM-PIM/AiM platforms across multiple contemporary DNN workloads. Compared with GEMM-based inference on DRAM-PIMs, PIM-DL achieves 22.6×~37.1× speedup. Compared with CPU/GPU-based inference, PIM-DL achieves up to 3.54×/1.20× speedup.
@inproceedings{LiZWYCYL024,
title = {PIM-DL: Expanding the Applicability of Commodity DRAM-PIMs for Deep Learning via Algorithm-System Co-Optimization},
author = {Cong Li and Zhe Zhou and Yang Wang and Fan Yang and Ting Cao and Mao Yang and Yun Liang and Guangyu Sun},
booktitle = {ASPLOS},
year = {2024},
doi = {10.1145/3620665.3640376}
}