一句话：提出 Prague，旨在让异步去中心化训练兼具 All-Reduce 的同构高性能与 AD-PSGD 的异构容忍性。
分布式深度学习训练通常采用 All-Reduce 作为数据并行的同步机制，因为它在同构环境中性能高；但其性能受最慢 worker 限制，因此在异构环境中显著变慢。AD-PSGD 是一种新提出的同步方法，具有数值收敛快和异构容忍性，但存在死锁和高同步开销。本文标题提出 Prague，试图结合二者优点：既像 All-Reduce 在同构环境下高性能，又像 AD-PSGD 具备良好异构容忍性，用于异步去中心化训练。摘要未给出具体机制与实验结果。
Distributed deep learning training usually adopts All-Reduce as the synchronization mechanism for data parallel algorithms due to its high performance in homogeneous environment. However, its performance is bounded by the slowest worker among all workers. For this reason, it is significantly slower in heterogeneous settings. AD-PSGD, a newly proposed synchronization method which provides numerically fast convergence and heterogeneity tolerance, suffers from deadlock issues and high synchronization overhead. Is it possible to get the best of both worlds --- designing a distributed training method that has both high performance like All-Reduce in homogeneous environment and good heterogeneity tolerance like AD-PSGD?