一句话：提出首个针对 LLM 全部非线性运算的插件式 CGRA 加速器及其编译工具链，推理最高提速 1.86×。
大语言模型中的非线性运算显著拉高推理延迟，且对加速器的效率、灵活性与用户自定义精度提出了新的要求。作者分析认为，粗粒度可重构阵列（CGRA）能够在性能与灵活性之间取得适合领域专用负载的平衡。据此提出 PICACHU：一种插件式 CGRA 加速器，通过定制算法与专用编译器工具链高效处理非线性运算。该工作是首个覆盖 LLM 内全部非线性运算、并将 CGRA 作为插件式加速器用于 LLM 推理的方案。评估表明，PICACHU 相较此前最先进的加速器在 LLM 推理上分别取得 1.86× 与 1.55× 的加速。
Large language models (LLMs) have revolutionized natural language processing (NLP) domain by achieving state-of-the-art performance across a range of benchmarks. However, nonlinear operations in LLMs significantly contribute to inference latency and present unique challenges that have not been encountered previously. Addressing these challenges requires accelerators that combine efficiency, flexibility, and support for user-defined precision. Our analysis reveals that Coarse-Grained Reconfigurable Arrays (CGRAs) provide an effective solution, offering a balance of performance and flexibility tailored to domain-specific workloads. This paper introduces PICACHU, a plug-in coarse-grained reconfigurable accelerator tailored to efficiently handle nonlinear operations by using custom algorithms and a dedicated compiler toolchain. PICACHU is the first to target all nonlinear operations within LLMs and to consider CGRA as a plug-in accelerator for LLM inference. Our evaluation shows that PICACHU achieves speedups of 1.86× and 1.55× over prior state-of-the-art accelerators in LLM inference.
@inproceedings{QinXT0Z25,
title = {PICACHU: Plug-In CGRA Handling Upcoming Nonlinear Operations in LLMs},
author = {Jiajun Qin and Tianhua Xia and Cheng Tan and Jeff Zhang and Sai Qian Zhang},
booktitle = {ASPLOS},
year = {2025},
doi = {10.1145/3676641.3716013}
}