一句话：提出 Astraea，通过微服务部署策略与自适应 GPU 通信框架，在满足 QoS 的同时把峰值负载提升最多 82.3%。
多阶段、面向用户的 GPU 应用如今常以微服务形式实现，但已有工作因通信模式与共享资源竞争方式不同，难以保障这类 GPU 微服务的服务质量（QoS）。本文提出 Astraea 来管理 GPU 微服务，并显式考虑上述两类因素。Astraea 包含两部分：一是微服务部署策略，目标是在满足所需 QoS 的前提下最大化系统可支撑的峰值服务负载；二是可自动伸缩的 GPU 通信框架，它根据当前硬件拓扑与微服务所在位置自适应地切换微服务之间的通信方式，并采用基于全局内存的技术来减少 GPU 内通信开销。评估表明，与最先进的方案相比，Astraea 在达到预期的 99% 尾延迟目标的同时，将可支撑的峰值负载最多提升 82.3%。
Multi-stage user-facing applications on GPUs are widely-used nowa- days, and are often implemented to be microservices. Prior re- search works are not applicable to ensuring QoS of GPU-based microservices due to the different communication patterns and shared resource contentions. We propose Astraea to manage GPU microservices considering the above factors. In Astraea, a microser- vice deployment policy is used to maximize the supported peak service load while ensuring the required QoS. To adaptively switch the communication methods between microservices according to different deployments, we propose an auto-scaling GPU communi- cation framework. The framework automatically scales based on the currently used hardware topology and microservice location, and adopts global memory-based techniques to reduce intra-GPU communication. Astraea increases the supported peak load by up to 82.3% while achieving the desired 99%-ile latency target compared with state-of-the-art solutions.
@inproceedings{ZhangCFZHLG22,
title = {Astraea: towards QoS-aware and resource-efficient multi-stage GPU services},
author = {Wei Zhang and Quan Chen and Kaihua Fu and Ningxin Zheng and Zhiyi Huang and Jingwen Leng and Minyi Guo},
booktitle = {ASPLOS},
year = {2022},
doi = {10.1145/3503222.3507721}
}