一句话：提出 FlashMem，用按需权重流式加载与 2.5D 纹理内存优化移动 GPU 上大模型和多 DNN 推理。
现代深度神经网络规模与复杂度不断增长，在内存和算力受限的移动 GPU 上进行端侧推理面临严峻挑战。现有 DNN 加速框架主要采用权重预加载策略，在移动 GPU 执行前将全部模型参数载入内存，难以适应包含超大模型或连续执行多个不同模型的现代 DNN 工作负载。本文提出 FlashMem，一个内存流式框架，旨在高效执行大规模现代 DNN 与多 DNN 工作负载，同时最小化内存消耗并降低推理延迟。FlashMem 不进行完整权重预加载，而是静态确定模型加载调度，并按需动态流式加载，同时利用 2.5D 纹理内存减少数据变换、提升执行效率。在 11 个模型上的实验表明，与现有框架相比，FlashMem 实现 2.0× 至 8.4× 内存降低和 1.7× 至 75.0× 加速，使资源受限的移动 GPU 能够高效执行大规模模型并支持多 DNN 工作负载。
The increasing size and complexity of modern deep neural networks (DNNs) pose significant challenges for on-device inference on mobile GPUs, with limited memory and computational resources. Existing DNN acceleration frameworks primarily deploy a weight preloading strategy, where all model parameters are loaded into memory before execution on mobile GPUs. We posit that this approach is not adequate for modern DNN workloads that comprise very large model(s) and possibly execution of several distinct models in succession. In this work, we introduce FlashMem, a memory streaming framework designed to efficiently execute large-scale modern DNNs and multi-DNN workloads while minimizing memory consumption and reducing inference latency. Instead of fully preloading weights, FlashMem statically determines model loading schedules and dynamically streams them on demand, leveraging 2.5D texture memory to minimize data transformations and improve execution efficiency. Experimental results on 11 models demonstrate that FlashMem achieves 2.0× to 8.4× memory reduction and 1.7× to 75.0× speedup compared to existing frameworks, enabling efficient execution of large-scale models and multi-DNN support on resource-constrained mobile GPUs.
DOI 原文 · arXiv · PDF（开放获取） · DBLP
@inproceedings{abs-2602-15379,
title = {FlashMem: Supporting Modern DNN Workloads on Mobile with GPU Memory Hierarchy Optimizations},
author = {Zhihao Shu and Md Musfiqur Rahman Sanim and Hangyu Zheng and Kunxiong Zhu and Miao Yin and Gagan Agrawal and Wei Niu},
booktitle = {ASPLOS},
year = {2026},
doi = {10.1145/3779212.3790164}
}