一句话：RAP 在 DLRM 训练期间自动复用 GPU 剩余算力做输入预处理，端到端吞吐平均提速 1.99 倍。
面向新用户的在线推荐要求用新生成的数据持续重训深度推荐模型（DLRM）。现有方案依赖数百个 CPU 节点完成输入预处理，其功耗甚至超过 GPU 训练部分。为此，作者提出 RAP：一个支持资源感知自动 GPU 共享的端到端 DLRM 训练框架，在不增加额外资源的前提下，把 DLRM 训练期间剩余的 GPU 算力用于输入预处理。RAP 使用协同运行代价模型评估各类输入预处理操作的开销，并设计资源感知的水平融合技术，根据 GPU 可用性自适应合并较小的 kernel，从而避免干扰 DLRM 训练；同时用启发式搜索联合优化输入预处理图映射与协同运行调度，最大化端到端训练吞吐。评测显示，RAP 相较基于 GPU 的顺序式输入预处理基线平均加速 1.99 倍，其端到端训练吞吐仅比无输入预处理开销的理想情况低 3.24%。
Ensuring high-quality recommendations for newly onboarded users requires the continuous retraining of Deep Learning Recommendation Models (DLRMs) with freshly generated data. To serve the online DLRM retraining, existing solutions use hundreds of CPU computing nodes designated for input preprocessing, causing significant power consumption that surpasses even the power usage of GPU trainers. To this end, we propose RAP, an end-to-end DLRM training framework that supports Resource-aware Automated GPU sharing for DLRM input Preprocessing and Training. The core idea of RAP is to accurately capture the remaining GPU computing resources during DLRM training for input preprocessing, achieving superior training efficiency without requiring additional resources. Specifically, RAP utilizes a co-running cost model to efficiently assess the costs of various input preprocessing operations, and it implements a resource-aware horizontal fusion technique that adaptively merges smaller kernels according to GPU availability, circumventing any interference with DLRM training. In addition, RAP leverages a heuristic searching algorithm that jointly optimizes both the input preprocessing graph mapping and the co-running schedule to maximize the end-to-end DLRM training throughput. The comprehensive evaluation shows that RAP achieves 1.99× speedup on average over the sequential GPU-based DLRM input preprocessing baseline. In addition, the end-to-end training throughput of RAP is only 3.24% lower than the ideal case, which has no input preprocessing overhead.
@inproceedings{WangWDZ0D24,
title = {RAP: Resource-aware Automated GPU Sharing for Multi-GPU Recommendation Model Training and Input Preprocessing},
author = {Zheng Wang and Yuke Wang and Jiaqi Deng and Da Zheng and Ang Li and Yufei Ding},
booktitle = {ASPLOS},
year = {2024},
doi = {10.1145/3620665.3640406}
}