一句话：提出由 GPU 编排的 GPU 显存-主机内存-SSD 三级内存分层，并用重用预测进行页面放置/旁路。
大数据集处理要求 GPU 直接访问仅 SSD 可承载的容量，但现有方案要么由主机 CPU 软件栈中转（Dragon、HMM），吞吐不足；要么经 NVMe 队列直连 SSD（BaM），无法利用主机内存作为中间层降低延迟。论文设计并实现 GMT：由 GPU 编排 GPU 内存、主机内存与 SSD 构成的三级层次，让 GPU 负责带宽/延迟敏感的大部分传输；并且不盲目在驱逐时把页面换到主机内存，而是用基于重用预测的插入策略进行有选择的页面放置或旁路。真实平台上对多种访存特征的 GPU 应用评测显示，GMT 相比 BaM 提升 50%，相比 HMM 提升超过 350%。
As the demand for processing larger datasets increases, GPUs need to reach deeper into their (memory) hierarchy to directly access capacities that only storage systems (SSDs) can hold. However, the state-of-the-art mechanisms to reach storage either employ software stacks running on the host CPUs as intermediaries (e.g. Dragon, HMM), which has been noted to perform poorly and not able to meet the throughput needs of GPU cores, or directly access SSDs through NVMe queues (BaM) which does not benefit from lower latencies that may be possible by having the host memory as an intermediate tier. This paper presents the design and implementation of GPU Memory Tiering (GMT) by implementing a GPU-orchestrated 3-tier hierarchy comprising GPU memory, host memory and SSDs, where the GPU orchestrates most of the transfers that are bandwidth/latency sensitive. Additionally, it is important to not blindly transfer pages from the GPU memory to host memory upon an eviction, and GMT employs a reuse-prediction based practical insertion policy to perform discretionary page placement/bypass. An implementation and evaluation on an actual platform demonstrates that GMT performs 50% better than the state-of-the-art 2-tier strategy (BaM) and over 350% better than the state-of-the-art 3-tier strategy that is orchestrated by host CPUs (HMM), over a number of GPU applications with diverse memory access characteristics.
@inproceedings{ChangHSMQH24,
title = {GMT: GPU Orchestrated Memory Tiering for the Big Data Era},
author = {Chia-Hao Chang and Jihoon Han and Anand Sivasubramaniam and Vikram Sharma Mailthody and Zaid Qureshi and Wen-Mei Hwu},
booktitle = {ASPLOS},
year = {2024},
doi = {10.1145/3620666.3651353}
}