一句话：提出NPU-PIM统一内存加速器IANUS及PIM访问调度，以加速LLM端到端推理。
论文提出IANUS，一个面向大语言模型端到端推理的NPU-PIM集成加速器。它结合NPU高计算吞吐与PIM高有效内存带宽，采用统一主存系统，使PIM内存既用于PIM运算又作为NPU主存，以提高容量利用率并减少数据搬移。针对普通访存与PIM计算不能同时进行的挑战，提出PIM Access Scheduling，统一调度访存、PIM计算及PIM/NPU工作负载映射。仿真显示GPT-2上平均较NVIDIA A100 GPU和当时最先进加速器分别提升6.2倍与3.2倍，并用商用PIM、NPU和FPGA控制器原型验证可行性。
Accelerating end-to-end inference of transformer-based large language models (LLMs) is a critical component of AI services in datacenters. However, the diverse compute characteristics of LLMs' end-to-end inference present challenges as previously proposed accelerators only address certain operations or stages (e.g., self-attention, generation stage, etc.). To address the unique challenges of accelerating end-to-end inference, we propose IANUS - Integrated Accelerator based on NPU-PIM Unified Memory System. IANUS is a domain-specific system architecture that combines a Neural Processing Unit (NPU) with a Processing-in-Memory (PIM) to leverage both the NPU's high computation throughput and the PIM's high effective memory bandwidth. In particular, IANUS employs a unified main memory system where the PIM memory is used both for PIM operations and for NPU's main memory. The unified main memory system ensures that memory capacity is efficiently utilized and the movement of shared data between NPU and PIM is minimized. However, it introduces new challenges since normal memory accesses and PIM computations cannot be performed simultaneously. Thus, we propose novel PIM Access Scheduling that manages not only the scheduling of normal memory accesses and PIM computations but also workload mapping across the PIM and the NPU. Our detailed simulation evaluations show that IANUS improves the performance of GPT-2 by 6.2× and 3.2×, on average, compared to the NVIDIA A100 GPU and the state-of-the-art accelerator. As a proof-of-concept, we develop a prototype of IANUS with a commercial PIM, NPU, and an FPGA-based PIM controller to demonstrate the feasibility of IANUS.
DOI 原文 · arXiv · PDF（开放获取） · DBLP
@inproceedings{abs-2410-15008,
title = {IANUS: Integrated Accelerator based on NPU-PIM Unified Memory System},
author = {Minseok Seo and Xuan Truong Nguyen and Seok Joong Hwang and Yongkee Kwon and Guhyun Kim and Chanwook Park and Ilkon Kim and Jaehan Park and Jeongbin Kim and Woojae Shin and Jongsoon Won and Haerang Choi and Kyuyoung Kim and Daehan Kwon and Chunseok Jeong and Sangheon Lee and Yongseok Choi and Wooseok Byun and Seungcheol Baek and Hyuk-Jae Lee and John Kim},
booktitle = {ASPLOS},
year = {2024},
doi = {10.1145/3620666.3651324}
}