一句话：提出 DeepUM，利用 CUDA 统一内存与相关性预取，实现 DNN 的 GPU 显存超分并隐藏页迁移开销。
深度神经网络规模持续增大，对 GPU 显存需求剧增。本文提出 DeepUM 框架，利用 CUDA 统一内存（UM）实现 GPU 显存超分，使超出物理显存的大模型仍可运行。针对 UM 页错误触发的页迁移开销，DeepUM 引入新的相关性预取技术自动、透明地预取张量页，并设计两种优化以减少 GPU 页错误处理时间。作者在 MLPerf、PyTorch examples 和 Hugging Face 的九个大规模 DNN 上评估，并与六种先进的 GPU 内存交换方法比较。结果表明 DeepUM 能有效支持显存超分，且能处理其他方法无法容纳的更大模型。
Deep neural networks (DNNs) are continuing to get wider and deeper. As a result, it requires a tremendous amount of GPU memory and computing power. In this paper, we propose a framework called DeepUM that exploits CUDA Unified Memory (UM) to allow GPU memory oversubscription for DNNs. While UM allows memory oversubscription using a page fault mechanism, page migration introduces enormous overhead. DeepUM uses a new correlation prefetching technique to hide the page migration overhead. It is fully automatic and transparent to users. We also propose two optimization techniques to minimize the GPU fault handling time. We evaluate the performance of DeepUM using nine large-scale DNNs from MLPerf, PyTorch examples, and Hugging Face and compare its performance with six state-of-the-art GPU memory swapping approaches. The evaluation result indicates that DeepUM is very effective for GPU memory oversubscription and can handle larger models that other approaches fail to handle.