一句话：提出基于物品共现图与部分和记忆化的算法-系统协同设计，加速推荐模型稀疏嵌入层推理。
推荐模型的稀疏嵌入层受限于高内存带宽需求，现有异构内存设计与部分嵌入和记忆化方法要么仅针对少量嵌入，要么处理开销过高，难以随现代嵌入表规模扩展。GRACE 提出轻量可扩展的算法-系统协同设计：用物品共现图（ICG）可扩展地记录共现关系，并通过系统感知的 ICG 聚类算法发现任意长度的频繁访问物品组合，计算并记忆化其部分和；高频部分和存入软件管理的缓存空间，配合缓存数据布局与低成本地址计算逻辑，降低内存流量并提升稀疏特征计算吞吐。评估表明，GRACE 相比 SPACE 和 MERCI 分别取得 1.5 倍和 1.4 倍加速。
The high memory bandwidth demand of sparse embedding layers continues to be a critical challenge in scaling the performance of recommendation models. While prior works have exploited heterogeneous memory system designs and partial embedding sum memoization techniques, they offer limited benefits. This is because prior designs either target a very small subset of embeddings to simplify their analysis or incur a high processing cost to account for all embeddings, which does not scale with the large sizes of modern embedding tables. This paper proposes GRACE-a lightweight and scalable graph-based algorithm-system co-design framework to significantly improve the embedding layer performance of recommendation models. GRACE proposes a novel Item Co-occurrence Graph (ICG) that scalably records item co-occurrences. GRACE then presents a new system-aware ICG clustering algorithm to find frequently accessed item combinations of arbitrary lengths to compute and memoize their partial sums. High-frequency partial sums are stored in a software-managed cache space to reduce memory traffic and improve the throughput of computing sparse features. We further present a cache data layout and low-cost address computation logic to efficiently lookup item embeddings and their partial sums. Our evaluation shows that GRACE significantly outperforms the state-of-the-art techniques SPACE and MERCI by 1.5x and 1.4x, respectively.
@inproceedings{paperbot10094,
title = {GRACE: A Scalable Graph-Based Approach to Accelerating Recommendation Model Inference},
author = {Haojie Ye and Sanketh Vedula and Yuhan Chen and Yichen Yang and Alex Bronstein and Ronald Dreslinski and Trevor Mudge and Nishil Talati},
booktitle = {ASPLOS},
year = {2023},
doi = {10.1145/3582016.3582029}
}