一句话：提出 DNNMem，一种在运行前精确估算深度学习模型 GPU 内存消耗的解析式工具。
深度学习任务常因 GPU 内存耗尽而失败，而开发者在执行前难以知道模型所需显存。本文提出 DNNMem，采用解析式估算方法，系统计算计算图与深度学习框架运行时两部分的 GPU 内存消耗，从而在训练或部署前预测显存占用。作者在 TensorFlow、PyTorch 和 MXNet 三个主流框架上，使用 5 个真实代表性模型及不同超参数进行实验。结果表明 DNNMem 能有效估算 GPU 内存消耗，有助于避免因架构或超参数选择不当导致的 OOM 失败，减少计算资源浪费并提升开发效率。
Deep learning (DL) has been increasingly adopted by a variety of software-intensive systems. Developers mainly use GPUs to accelerate the training, testing, and deployment of DL models. However, the GPU memory consumed by a DL model is often unknown to them before the DL job executes. Therefore, an improper choice of neural architecture or hyperparameters can cause such a job to run out of the limited GPU memory and fail. Our recent empirical study has found that many DL job failures are due to the exhaustion of GPU memory. This leads to a horrendous waste of computing resources and a significant reduction in development productivity. In this paper, we propose DNNMem, an accurate estimation tool for GPU memory consumption of DL models. DNNMem employs an analytic estimation approach to systematically calculate the memory consumption of both the computation graph and the DL framework runtime. We have evaluated DNNMem on 5 real-world representative models with different hyperparameters under 3 mainstream frameworks (TensorFlow, PyTorch, and MXNet). Our extensive experiments show that DNNMem is effective in estimating GPU memory consumption.
DOI 原文 ·
@inproceedings{paperbot5797,
title = {Estimating GPU memory consumption of deep learning models},
author = {Yanjie Gao and Yu Liu and Hongyu Zhang and Zhengxian Li and Yonghao Zhu and Haoxiang Lin and Mao Yang},
booktitle = {FSE},
year = {2020},
doi = {10.1145/3368089.3417050}
}