一句话：提出 ElasticFlow 弹性无服务器分布式深度学习训练平台，通过按截止期准入与动态资源分配保障训练性能。
论文提出 ElasticFlow，一个面向分布式深度学习的弹性无服务器训练平台。用户只需指定 DNN 模型、超参数和作业截止期，无需指定 GPU 数量或占用时长，从而免除底层手工资源管理。分布式训练中，吞吐随 GPU 数非线性扩展，且扩展效率受 worker 放置影响。作者提出 Minimum Satisfactory Share 刻画作业满足截止期所需资源并据此准入控制；用基于收益递减的贪心算法动态分配资源；用 buddy allocation 放置 worker 以消除拓扑影响。在 128 块 GPU 集群上，ElasticFlow 使按期完成作业数较已有方案提升 1.46–7.65 倍。
This paper proposes ElasticFlow, an elastic serverless training platform for distributed deep learning. ElasticFlow provides a serverless interface with two distinct features: (i) users specify only the deep neural network (DNN) model and hyperparameters for a job, but not the number of GPUs; (ii) users specify the deadline for a job, but not the amount of time to occupy GPUs. In contrast to existing server-centric platforms, ElasticFlow provides performance guarantees in terms of meeting deadlines while alleviating tedious, low-level, and manual resource management for deep learning developers. The characteristics of distributed training introduce two challenges. First, the training throughput scales non-linearly with the number of GPUs. Second, the scaling efficiency is affected by worker placement. To address these challenges, we propose Minimum Satisfactory Share to capture the resource usage of training jobs to meet deadlines, and ElasticFlow performs admission control based on it. We develop a greedy algorithm that dynamically allocates resources to admitted jobs based on diminishing returns. We apply buddy allocation to worker placement to eliminate the effect of topology. Evaluation results on a cluster of 128 GPUs show that ElasticFlow increases the number of jobs that can meet their deadlines by 1.46–7.65× compared to existing solutions.
@inproceedings{paperbot10024,
title = {ElasticFlow: An Elastic Serverless Training Platform for Distributed Deep Learning},
author = {Diandian Gu and Yihao Zhao and Yinmin Zhong and Yifan Xiong and Zhenhua Han and Peng Cheng and Fan Yang and Gang Huang and Xin Jin and Xuanzhe Liu},
booktitle = {ASPLOS},
year = {2023},
doi = {10.1145/3575693.3575721}
}