一句话：提出 MASK 框架，通过三项地址转换感知的缓存与内存管理机制，缓解多应用共享 GPU 时的 TLB 争用与性能损失。
现代 GPU 缺少 CPU 式的多应用并发支持，多应用共享 GPU 时，面向单应用的地址转换机制产生严重干扰，导致共享 TLB 频繁缺失，数百线程长期停顿，系统吞吐下降。作者分析出虚拟内存，尤其地址转换性能是主要瓶颈，提出 MASK 框架，包含基于 token 降低 TLB 争用、旁路提升缓存地址转换有效性、应用感知内存调度减少转换与数据请求干扰三项机制。评估显示，相比先进 GPU TLB，MASK 提升系统吞吐 57.8%、IPC 吞吐 43.4%，降低应用级不公平性 22.4%，并与无地址转换开销的理想 GPU 相差 23.2% 以内。
Graphics Processing Units (GPUs) exploit large amounts of threadlevel parallelism to provide high instruction throughput and to efficiently hide long-latency stalls. The resulting high throughput, along with continued programmability improvements, have made GPUs an essential computational resource in many domains. Applications from different domains can have vastly different compute and memory demands on the GPU. In a large-scale computing environment, to efficiently accommodate such wide-ranging demands without leaving GPU resources underutilized, multiple applications can share a single GPU, akin to how multiple applications execute concurrently on a CPU. Multi-application concurrency requires several support mechanisms in both hardware and software. One such key mechanism is virtual memory, which manages and protects the address space of each application. However, modern GPUs lack the extensive support for multi-application concurrency available in CPUs, and as a result suffer from high performance overheads when shared by multiple applications, as we demonstrate. We perform a detailed analysis of which multi-application concurrency support limitations hurt GPU performance the most. We find that the poor performance is largely a result of the virtual memory mechanisms employed in modern GPUs. In particular, poor address translation performance is a key obstacle to efficient GPU sharing. State-of-the-art address translation mechanisms, which were designed for single-application execution, experience significant inter-application interference when multiple applications spatially share the GPU. This contention leads to frequent misses in the shared translation lookaside buffer (TLB), where a single miss can induce long-latency stalls for hundreds of threads. As a result, the GPU often cannot schedule enough threads to successfully hide the stalls, which diminishes system throughput and becomes a first-order performance concern. Based on our analysis, we propose MASK, a new GPU framework that provides low-overhead virtual memory support for the concurrent execution of multiple applications. MASK consists of three novel address-translation-aware cache and memory management mechanisms that work together to largely reduce the overhead of address translation: (1) a token-based technique to reduce TLB contention, (2) a bypassing mechanism to improve the effectiveness of cached address translations, and (3) an application-aware memory scheduling scheme to reduce the interference between address translation and data requests. Our evaluations show that MASK restores much of the throughput lost to TLB contention. Relative to a state-of-the-art GPU TLB, MASK improves system throughput by 57.8%, improves IPC throughput by 43.4%, and reduces applicationlevel unfairness by 22.4%. MASK's system throughput is within 23.2% of an ideal GPU system with no address translation overhead.
DOI 原文 ·
@inproceedings{paperbot4247,
title = {MASK},
author = {Rachata Ausavarungnirun and Vance Miller and Joshua Landgraf and Saugata Ghose and Jayneel Gandhi and Adwait Jog and Christopher J. Rossbach and Onur Mutlu},
booktitle = {ASPLOS},
year = {2018},
doi = {10.1145/3173162.3173169}
}