一句话：通过解耦虚拟与物理内存分配，在保持 KV cache 虚拟内存连续性的同时缓解碎片，提升 LLM 服务吞吐。
PagedAttention 是 LLM 服务系统中常用的动态显存分配方案，可按需分配 GPU 内存以缓解 KV cache 碎片，从而提升批大小与吞吐。然而，它在运行时分配物理内存时把 KV cache 的虚拟内存布局从连续变为非连续，带来显著的编程与性能开销。本文提出 vAttention，在缓解物理内存碎片的同时保留 KV cache 的虚拟内存连续性。其核心思路是利用 CUDA 虚拟内存管理 API 将虚拟内存与物理内存的分配解耦，并针对 CUDA 虚拟内存支持的局限引入多项 LLM 专用优化。评估表明，vAttention 是比 PagedAttention 更简单、可移植且性能更优的替代方案：它开箱即用地支持多种 attention kernel，相比使用 FlashAttention-2 与 FlashInfer 的 PagedAttention kernel，LLM 服务吞吐最高提升 1.23 倍。
PagedAttention is a popular approach for dynamic memory allocation in LLM serving systems. It enables on-demand allocation of GPU memory to mitigate KV cache fragmentation - a phenomenon that crippled the batch size (and consequently throughput) in prior systems. However, in trying to allocate physical memory at runtime, PagedAttention ends up changing the virtual memory layout of the KV cache from contiguous to non-contiguous. Such a design leads to non-trivial programming and performance overheads. We present vAttention - an approach that mitigates fragmentation in physical memory while retaining the virtual memory contiguity of the KV cache. We achieve this by decoupling the allocation of virtual and physical memory using CUDA virtual memory management APIs. We also introduce various LLM-specific optimizations to address the limitations of CUDA virtual memory support. Overall, vAttention is a simpler, portable, and performant alternative to PagedAttention: it supports various attention kernels out-of-the-box and improves LLM serving throughput by up to 1.23× compared to the use of PagedAttention-based kernels of FlashAttention-2 and FlashInfer.
DOI 原文 · arXiv · PDF（开放获取） · DBLP
@inproceedings{abs-2405-04437,
title = {vAttention: Dynamic Memory Management for Serving LLMs without PagedAttention},
author = {Ramya Prabhu and Ajay Nayak and Jayashree Mohan and Ramachandran Ramjee and Ashish Panwar},
booktitle = {ASPLOS},
year = {2025},
doi = {10.1145/3669940.3707256}
}