一句话：提出 SoCFlow，首个能在 SoC 集群边缘服务器上高效、可扩展训练深度模型的框架。
论文关注由大量移动端 system-on-chip（SoC）构成的 SoC-Cluster 边缘服务器：其上负载多由用户触发，呈现明显潮汐现象，导致资源长期闲置。作者提出将深度学习训练任务共置到这些空闲周期上，但商用 SoC 集群存在网络瓶颈与处理器异构等固有不足。SoCFlow 为此设计两项技术：一是组内并行配合延迟聚合，使模型训练在不受网络瓶颈影响的前提下快速且可扩展；二是数据并行的混合精度训练算法，充分释放移动 SoC 异构处理器的算力。作者完整实现了 SoCFlow 并开展大量实验，结果表明其在保持收敛精度的同时，训练速度一致且显著优于所有基线，32 个 SoC 上收敛加速达 1.6–740 倍；在与 NVIDIA V100 相同功耗预算下取得相当训练速度，能耗降低 2.31–10.23 倍。
SoC-Cluster, a novel server architecture composed of massive mobile system-on-chips (SoCs), is gaining popularity in industrial edge computing due to its energy efficiency and compatibility with existing mobile applications. However, we observe that the deployed SoC-Cluster servers are not fully utilized, because the hosted workloads are mostly user-triggered and have significant tidal phenomena. To harvest the free cycles, we propose to co-locate deep learning tasks on them. We present SoCFlow, the first framework that can efficiently train deep learning models on SoC-Cluster. To deal with the intrinsic inadequacy of commercial SoC-Cluster servers, SoCFlow incorporates two novel techniques: (1) the group-wise parallelism with delayed aggregation that can train deep learning models fast and scalably without being influenced by the network bottleneck; (2) the data-parallel mixed-precision training algorithm that can fully unleash the heterogeneous processors' capability of mobile SoCs. We have fully implemented SoCFlow and demonstrated its effectiveness through extensive experiments. The experiments show that SoCFlow significantly and consistently outperforms all baselines regarding the training speed while preserving the convergence accuracy, e.g., 1.6×--740× convergence speedup with 32 SoCs. Compared to commodity GPU (NVIDIA V100) under the same power budget, SoCFlow achieves comparable training speed but reduces energy consumption by 2.31×--10.23× with the same convergence accuracy.
@inproceedings{XuXLZHJL24,
title = {SoCFlow: Efficient and Scalable DNN Training on SoC-Clustered Edge Servers},
author = {Daliang Xu and Mengwei Xu and Chiheng Lou and Li Zhang and Gang Huang and Xin Jin and Xuanzhe Liu},
booktitle = {ASPLOS},
year = {2024},
doi = {10.1145/3617232.3624847}
}