一句话：提出 RecShard，基于数据分布与内存带宽对推荐模型嵌入表细粒度分片和放置，显著提升训练吞吐。
深度推荐模型（DLRM）中的嵌入表（EMB）规模巨大，访问模式不均，且因哈希表特性常出现生日悖论而利用率低，导致训练受限于分级内存带宽。RecShard 基于训练数据分布、模型特征与内存层级带宽，对 EMB 进行细粒度分片和放置，为每组 EMB 选择最优 sharding 策略。实验表明，在容量受限的 DLRM 上，RecShard 将 EMB 训练吞吐平均提升 6 倍以上，负载均衡提升超过 12 倍，对慢速内存的访问减少超过 87 倍。
We propose RecShard, a fine-grained embedding table (EMB) partitioning and placement technique for deep learning recommendation models (DLRMs). RecShard is designed based on two key observations. First, not all EMBs are equal, nor all rows within an EMB are equal in terms of access patterns. EMBs exhibit distinct memory characteristics, providing performance optimization opportunities for intelligent EMB partitioning and placement across a tiered memory hierarchy. Second, in modern DLRMs, EMBs function as hash tables. As a result, EMBs display interesting phenomena, such as the birthday paradox, leaving EMBs severely under-utilized. RecShard determines an optimal EMB sharding strategy for a set of EMBs based on training data distributions and model characteristics, along with the bandwidth characteristics of the underlying tiered memory hierarchy. In doing so, RecShard achieves over 6 times higher EMB training throughput on average for capacity constrained DLRMs. The throughput increase comes from improved EMB load balance by over 12 times and from the reduced access to the slower memory by over 87 times.
DOI 原文 · arXiv · PDF（开放获取） · DBLP
@inproceedings{abs-2201-10095,
title = {RecShard: statistical feature-based memory optimization for industry-scale neural recommendation},
author = {Geet Sethi and Bilge Acun and Niket Agarwal and Christos Kozyrakis and Caroline Trippel and Carole-Jean Wu},
booktitle = {ASPLOS},
year = {2022},
doi = {10.1145/3503222.3507777}
}