一句话：提出MERCI记忆化框架，聚类并缓存相关嵌入的部分聚合，减少DNN嵌入归约内存访问，在商用硬件提升吞吐与能效。
带嵌入层的深度神经网络广泛用于建模实体间复杂关系。嵌入归约将多个嵌入向量聚合为一个，但其运行时间主要花在从内存读取嵌入上，是典型的内存带宽瓶颈。现有加速工作依赖硬件修改或新兴内存技术，难以部署在商用硬件上。本文提出 MERCI（Memoization for Embedding Reduction with ClusterIng），一种面向高效嵌入归约的记忆化框架：它通过聚类识别相关嵌入，记忆化其部分聚合结果，并以低开销检索，从而减少重复内存访问。在真实机器上，MERCI 将内存访问减少 44%（29%），吞吐提升 102%（74%），能耗降低 40.2%（28.6%），代价是额外 8×（1×）内存占用。
Deep neural networks (DNNs) with embedding layers are widely adopted to capture complex relationships among entities within a dataset. Embedding layers aggregate multiple embeddings — a dense vector used to represent the complicated nature of a data feature— into a single embedding; such operation is called embedding reduction. Embedding reduction spends a significant portion of its runtime on reading embeddings from memory and thus is known to be heavily memory-bandwidth-bound. Recent works attempt to accelerate this critical operation, but they often require either hardware modifications or emerging memory technologies, which makes it hardly deployable on commodity hardware. Thus, we propose MERCI, Memoization for Embedding Reduction with ClusterIng, a novel memoization framework for efficient embedding reduction. MERCI provides a mechanism for memoizing partial aggregation of correlated embeddings and retrieving the memoized partial result at a low cost. MERCI substantially reduces the number of memory accesses by 44% (29%), leading to 102% (74%) throughput improvement on real machines and 40.2% (28.6%) energy savings at the expense of 8×(1×) additional memory usage.
DOI 原文 ·
@inproceedings{paperbot6098,
title = {MERCI: efficient embedding reduction on commodity hardware via sub-query memoization},
author = {Yejin Lee and Seong Hoon Seo and Hyunji Choi and Hyoung Uk Sul and Soosung Kim and Jae W. Lee and Tae Jun Ham},
booktitle = {ASPLOS},
year = {2021},
doi = {10.1145/3445814.3446717}
}