一句话：基于微软内部平台 400 个真实作业，实证刻画深度学习作业低 GPU 利用率的表现、根因与修复方式。
企业开发者通常在共享的多租户平台上提交并运行深度学习作业，平台配置大量 GPU 以加速训练，但部分作业的 GPU 利用率很低，造成资源浪费与开发效率下降。本文对微软内部深度学习平台上 400 个真实作业（平均 GPU 利用率不超过 50%）开展实证研究，细致审查作业元数据、执行日志、运行时指标、脚本与程序，共识别出 706 个低利用率问题，并归纳其常见根因、给出对应修复方案。主要发现包括：低利用率源于 GPU 计算不足以及非 GPU 任务造成的中断；约 46.03% 的问题归因于数据操作；45.18% 与深度学习模型相关，且在训练与评估阶段均有出现；84.99% 的问题可通过少量代码或脚本修改修复。基于上述结果，作者进一步提出了帮助开发者在云平台上更好利用 GPU 的研究方向。
Deep learning plays a critical role in numerous intelligent software applications. Enterprise developers submit and run deep learning jobs on shared, multi-tenant platforms to efficiently train and test models. These platforms are typically equipped with a large number of graphics processing units (GPUs) to expedite deep learning computations. However, certain jobs exhibit rather low utilization of the allocated GPUs, resulting in substantial resource waste and reduced development productivity. This paper presents a comprehensive empirical study on low GPU utilization of deep learning jobs, based on 400 real jobs (with an average GPU utilization of 50% or less) collected from Microsoft's internal deep learning platform. We discover 706 low-GPU-utilization issues through meticulous examination of job metadata, execution logs, runtime metrics, scripts, and programs. Furthermore, we identify the common root causes and propose corresponding fixes. Our main findings include: (1) Low GPU utilization of deep learning jobs stems from insufficient GPU computations and interruptions caused by non-GPU tasks; (2) Approximately half (46.03%) of the issues are attributed to data operations; (3) 45.18% of the issues are related to deep learning models and manifest during both model training and evaluation stages; (4) Most (84.99%) low-GPU-utilization issues could be fixed with a small number of code/script modifications. Based on the study results, we propose potential research directions that could help developers utilize GPUs better in cloud-based platforms.
@inproceedings{paperbot11400,
title = {An Empirical Study on Low GPU Utilization of Deep Learning Jobs},
author = {Yanjie Gao and Yichen He and Xinze Li and Bo Zhao and Haoxiang Lin and Yoyo Liang and Jing Zhong and Hongyu Zhang and Jingzhou Wang and Yonghua Zeng and Keli Gui and Jie Tong and Mao Yang},
booktitle = {ICSE},
year = {2024},
doi = {10.1145/3597503.3639232}
}