一句话：提出 PAPI 异构架构，通过在线内核刻画动态调度计算受限与内存受限内核，加速 LLM 解码。
LLM 解码是生成输出 token 的关键且耗时阶段，其内核既包含计算受限型也包含内存受限型。已有工作通常静态地识别并把不同内核映射到由 PIM 单元与 GPU 等计算中心加速器组成的异构架构上。作者观察到，随着参数调整以适应用户或系统需求，解码内核的特性（例如是否内存受限）会动态变化，导致静态映射次优；同时内存受限内核之间也存在高度异构性，使“一刀切”式的 PIM 设计效率低下。为此，论文提出 PAPI：一种支持 PIM 的异构架构，包含两项关键机制：（1）在线内核刻画，在运行时将内核动态调度到最合适的硬件单元；（2）由计算中心处理单元（GPU）与具备不同计算能力的混合 PIM 单元协同编排的异构计算系统。在 LLaMA-65B、GPT-3 66B 与 GPT-3 175B 上的实验表明，PAPI 相对于最先进的 GPU+PIM 异构 LLM 加速器与纯 PIM LLM 加速器分别取得 1.8 倍与 11.1 倍加速。
Large language models (LLMs) are widely used for natural language understanding and text generation. An LLM model relies on a time-consuming step called LLM decoding to generate output tokens. Several prior works focus on improving the performance of LLM decoding using parallelism techniques, such as batching and speculative decoding. State-of-the-art LLM decoding has both compute-bound and memory-bound kernels. Some prior works statically identify and map these different kernels to a heterogeneous architecture consisting of both processing-in-memory (PIM) units and computation-centric accelerators (e.g., GPUs). We observe that characteristics of LLM decoding kernels (e.g., whether or not a kernel is memory-bound) can change dynamically due to parameter changes to meet user and/or system demands, making (1) static kernel mapping to PIM units and computation-centric accelerators suboptimal, and (2) one-size-fits-all approach of designing PIM units inefficient due to a large degree of heterogeneity even in memory-bound kernels. In this paper, we aim to accelerate LLM decoding while considering the dynamically changing characteristics of the kernels involved. We propose PAPI (PA rallel Decoding with PI M), a PIM-enabled heterogeneous architecture that exploits dynamic scheduling of compute-bound or memory-bound kernels to suitable hardware units. PAPI has two key mechanisms: (1) online kernel characterization to dynamically schedule kernels to the most suitable hardware units at runtime and (2) a PIM-enabled heterogeneous computing system that harmoniously orchestrates both computation-centric processing units (GPU) and hybrid PIM units with different computing capabilities. Our experimental results on three broadly-used LLMs (i.e., LLaMA-65B, GPT-3 66B, and GPT-3 175B) show that PAPI achieves 1.8× and 11.1× speedups over a state-of-the-art heterogeneous LLM accelerator (i.e., GPU and PIM) and a state-of-the-art PIM-only LLM accelerator, respectively.
DOI 原文 · arXiv · PDF（开放获取） · DBLP
@inproceedings{abs-2502-15470,
title = {PAPI: Exploiting Dynamic Parallelism in Large Language Model Decoding with a Processing-In-Memory-Enabled Computing System},
author = {Yintao He and Haiyu Mao and Christina Giannoula and Mohammad Sadrosadati and Juan Gómez-Luna and Huawei Li and Xiaowei Li and Ying Wang and Onur Mutlu},
booktitle = {ASPLOS},
year = {2025},
doi = {10.1145/3676641.3716009}
}