一句话：提出 GFS 抢占式调度框架，用需求预测与动态 spot 配额降低驱逐率与排队延迟。
大语言模型训练与推理的激增重塑了 GPU 使用模式，云厂商虽以 spot 实例承载低优先级（LP）任务来降低成本，但现有调度器仍面临高驱逐率与长排队时间。本文提出 GFS，一个抢占式调度框架，在提升高优先级（HP）任务 SLO 达成率的同时尽量减少对 LP 任务的抢占。GFS 由三部分构成：一是轻量级预测模型，预测不同租户的 GPU 需求以支持主动式资源管理；二是动态分配机制，在带保证时长约束下调整 LP 任务的 spot 配额；三是抢占式调度策略，优先保障 HP 任务并抑制对 LP 任务的冲击。作者通过真实系统实现与模拟两种方式评估：LP 任务驱逐率下降 33.0%，排队延迟减少 44.1%；在生产集群中 GPU 分配率最高提升 22.8%；在超过 10,000 块 GPU 的生产集群中每月带来约 459,715 美元的收益。
The surge in large language models (LLMs) has fundamentally reshaped the landscape of GPU usage patterns, creating an urgent need for more efficient management strategies. While cloud providers employ spot instances to reduce costs for low-priority (LP) tasks, existing schedulers still grapple with high eviction rates and lengthy queuing times. To address these limitations, we present GFS, a novel preemptive scheduling framework that enhances service-level objective (SLO) compliance for high-priority (HP) tasks while minimizing preemptions to LP tasks. Firstly, GFS utilizes a lightweight forecasting model that predicts GPU demand among different tenants, enabling proactive resource management. Secondly, GFS employs a dynamic allocation mechanism to adjust the spot quota for LP tasks with guaranteed durations. Lastly, GFS incorporates a preemptive scheduling policy that prioritizes HP tasks while minimizing the impact on LP tasks. We demonstrate the effectiveness of GFS through both real-world implementation and simulations. The results show that GFS reduces eviction rates by 33.0%, and cuts queuing delays by 44.1% for LP tasks. Furthermore, GFS enhances the GPU allocation rate by up to 22.8% in real production clusters. In a production cluster of more than 10,000 GPUs, GFS yields roughly $459,715 in monthly benefits.
DOI 原文 · arXiv · PDF（开放获取） · DBLP
@inproceedings{abs-2509-11134,
title = {GFS: A Preemption-aware Scheduling Framework for GPU Clusters with Predictive Spot Instance Management},
author = {Jiaang Duan and Shenglin Xu and Shiyou Qian and Dingyu Yang and Kangjin Wang and Chenzhi Liao and Yinghao Yu and Qin Hua and Hanwen Hu and Qi Wang and Wenchao Wu and Dongqing Bao and Tianyu Lu and Jian Cao and Guangtao Xue and Guodong Yang and Liping Zhang and Gang Chen},
booktitle = {ASPLOS},
year = {2025},
doi = {10.1145/3760250.3762231}
}