一句话：提出 FastGL，协同优化子图采样、内存 IO 与 GPU 计算，平均加速现有 GNN 训练框架。
基于采样的训练是扩展到十亿级节点/边大图 GNN 的常用方案，但现有框架在子图采样、内存 IO 和计算三个阶段均存在瓶颈。FastGL 同时优化这三个阶段：用 Match-Reorder 利用图结构中的数据重叠减少数据流量，在不增加 GPU 显存开销的前提下加速内存 IO；用 Memory-Aware 计算利用 GPU 内存层级缓解不规则访存；用 Fused-Map 降低采样过程中的同步开销。实验表明，FastGL 相比最先进框架 PyG、DGL 和 GNNLab 分别取得平均 11.8×、2.2× 和 1.5× 的加速。
Graph Neural Networks (GNNs) have shown great superiority on non-Euclidean graph data, achieving ground-breaking performance on various graph-related tasks. As a practical solution to train GNN on large graphs with billions of nodes and edges, the sampling-based training is widely adopted by existing training frameworks. However, through an in-depth analysis, we observe that the efficiency of existing sampling-based training frameworks is still limited due to the key bottlenecks lying in all three phases of sampling-based training, i.e., subgraph sample, memory IO, and computation. To this end, we propose FastGL, a GPU-efficient Framework for accelerating sampling-based training of GNN at Large scale by simultaneously optimizing all above three phases, taking into account both GPU characteristics and graph structure. Specifically, by exploiting the inherent overlap within graph structures, FastGL develops the Match-Reorder strategy to reduce the data traffic, which accelerates the memory IO without incurring any GPU memory overhead. Additionally, FastGL leverages a Memory-Aware computation method, harnessing the GPU memory's hierarchical nature to mitigate irregular data access during computation. FastGL further incorporates the Fused-Map approach aimed at diminishing the synchronization overhead during sampling. Extensive experiments demonstrate that FastGL can achieve an average speedup of 11.8×, 2.2× and 1.5× over the state-of-the-art frameworks PyG, DGL, and GNNLab, respectively. Our code is available at https://github.com/a1bc2def6g/fastgl-ae.
DOI 原文 · arXiv · PDF（开放获取） · DBLP
@inproceedings{ZhuWHLL024,
title = {FastGL: A GPU-Efficient Framework for Accelerating Sampling-Based GNN Training at Large Scale},
author = {Zeyu Zhu and Peisong Wang and Qinghao Hu and Gang Li and Xiaoyao Liang and Jian Cheng},
booktitle = {ASPLOS},
year = {2024},
doi = {10.1145/3622781.3674167}
}