一句话：提出基于ILP的Profile引导优化AutoTM，自动在DRAM与NVDIMM间迁移张量，降低DNN训练内存需求。
大规模神经网络训练面临内存容量瓶颈。Intel Optane DC PMM（NVDIMM）提供比SSD更高读带宽、比DRAM更低每比特成本，但访问性能低于DRAM，因此需要决定活跃张量应放在DRAM还是NVDIMM，以及何时迁移。本文利用深度神经网络计算图的静态特性，提出一种基于Profile引导优化和整数线性规划（ILP）的方法AutoTM，对活跃张量在DRAM与NVDIMM间的分配与移动进行最优调度。评估表明，该方法可将系统50%至80%的DRAM替换为PMM，而几何平均性能仅损失27.7%，显著优于first-touch NUMA的71.9%损失；对超大网络，其同步调度相对将DRAM作为硬件管理缓存还可获得约2倍性能。
Memory capacity is a key bottleneck for training large scale neural networks. Intel® Optane#8482; DC PMM (persistent memory modules) which are available as NVDIMMs are a disruptive technology that promises significantly higher read bandwidth than traditional SSDs at a lower cost per bit than traditional DRAM. In this work we show how to take advantage of this new memory technology to minimize the amount of DRAM required without compromising performance significantly. Specifically, we take advantage of the static nature of the underlying computational graphs in deep neural network applications to develop a profile guided optimization based on Integer Linear Programming (ILP) called AutoTM to optimally assign and move live tensors to either DRAM or NVDIMMs. Our approach can replace 50% to 80% of a system's DRAM with PMM while only losing a geometric mean 27.7% performance. This is a significant improvement over first-touch NUMA, which loses 71.9% of performance. The proposed ILP based synchronous scheduling technique also provides 2x performance over using DRAM as a hardware-controlled cache for very large networks.
@inproceedings{paperbot5397,
title = {AutoTM: Automatic Tensor Movement in Heterogeneous Memory Systems using Integer Linear Programming},
author = {Mark Hildebrand and Jawad Khan and Sanjeev Trika and Jason Lowe-Power and Venkatesh Akella},
booktitle = {ASPLOS},
year = {2020},
doi = {10.1145/3373376.3378465}
}