一句话:提出传输透明的网络内聚合方案,使分布式训练无需定制传输层即可利用 INA 加速。
现有网络内聚合(INA)方案将 all-reduce 操作卸载到网络交换机,以加速和扩展分布式训练。然而在端主机上,这些方案需要构建定制网络栈来替换传输层。这种面向 INA 的网络栈无法利用当前高性能的传输层实现,也给系统开发与运维带来复杂性。论文提出具有传输透明性的 INA 方案,使分布式训练能够在不替换传输层的情况下使用网络内聚合。摘要未报告具体实现与实验结果。
Recent In-Network Aggregation (INA) solutions offload the all-reduce operation onto network switches to accelerate and scale distributed training (DT). On end hosts, these solutions build custom network stacks to replace the transport layer. The INA-oriented network stack cannot take advantage of the state-of-the-art performant transport layer implementation, and also causes complexity in system development and operation.
@inproceedings{paperbot10063,
title = {In-Network Aggregation with Transport Transparency for Distributed Training},
author = {Shuo Liu and Qiaoling Wang and Junyi Zhang and Wenfei Wu and Qinliang Lin and Yao Liu and Meng Xu and Marco Canini and Ray C. C. Cheung and Jianfei He},
booktitle = {ASPLOS},
year = {2023},
doi = {10.1145/3582016.3582037}
}