一句话：TAPAS 利用历史温度/功耗数据与 SaaS 负载可调性，对 LLM 推理集群做热与功耗感知的放置、路由和重配置调度，降低 TCO 与限流事件。
生成式大模型推理需求激增，给云数据中心的热管理与功耗管理带来挑战。传统方法难以适配 LLM 推理毫秒级的细粒度执行阶段，因为各阶段的性能、温度与功耗特征差异明显；同时模型并行度、规模、量化等配置参数会在性能、温度、功耗与输出质量之间形成权衡，而云中 SaaS 与 IaaS 混部又造成可见性与灵活性不一致。为此作者提出 TAPAS，一个面向云中 LLM 推理集群的热感知与功耗感知框架。它增强冷却与功耗的超配能力，利用历史温度与功耗数据以及 SaaS 工作负载的可适应性，完成三件事：在冷却与功耗约束下放置新的 GPU 工作负载虚拟机、在 SaaS 虚拟机之间路由推理请求、以及重配置 SaaS 虚拟机以应对负载尖峰与冷却/功耗故障等紧急情况。在大规模 GPU 集群上的评估表明，该方法显著减少了热限流与功耗限流事件，提升了系统效率并降低了总拥有成本。
The rising demand for generative large language models (LLMs) poses challenges for thermal and power management in cloud datacenters. Traditional techniques are often inadequate for LLM inference due to the fine-grained, millisecond-scale execution phases, each with distinct performance, thermal, and power profiles. Additionally, LLM inference workloads are sensitive to various configuration parameters (e.g., model parallelism, size, and quantization) that involve trade-offs between performance, temperature, power, and output quality. Moreover, clouds often co-locate SaaS and IaaS workloads, each with different levels of visibility and flexibility. To address these challenges, we propose TAPAS, a thermal- and power-aware framework designed for LLM inference clusters in the cloud. TAPAS enhances cooling and power oversubscription capabilities, reducing the total cost of ownership (TCO) while effectively handling emergencies (e.g., cooling and power failures). TAPAS leverages historical temperature and power data, along with the adaptability of SaaS workloads, to: (1) efficiently place new GPU workload VMs within cooling and power constraints, (2) route LLM inference requests across SaaS VMs, and (3) reconfigure SaaS VMs to manage load spikes and emergency situations. Our evaluation on a large GPU cluster demonstrates significant reductions in thermal and power throttling events, boosting system efficiency.
DOI 原文 · arXiv · PDF（开放获取） · DBLP
@inproceedings{abs-2501-02600,
title = {TAPAS: Thermal- and Power-Aware Scheduling for LLM Inference in Cloud Platforms},
author = {Jovan Stojkovic and Chaojie Zhang and Íñigo Goiri and Esha Choukse and Haoran Qiu and Rodrigo Fonseca and Josep Torrellas and Ricardo Bianchini},
booktitle = {ASPLOS},
year = {2025},
doi = {10.1145/3676641.3716025}
}