paperbot · PL 论文追踪

全部论文 待读 收藏

P2 The ASPLOS 2025 / EuroSys 2025 Contest on Intra-Operator Parallelism for Distributed Deep Learning

ASPLOS 2025 / EuroSys 2025 分布式深度学习算子内并行竞赛
ASPLOS Proceedings of the 30th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 32025匹配度 3/10 · 该论文主题是分布式深度学习的算子内并行与模型划分竞赛回顾,属于机器学习系统与编译优化,与用户核心关注的静态分析、Agentic 程序、浮点/贝叶斯程序分析及 LLM 交叉方向无直接关联。虽然发布了新基准,但任务场景与用户研究画像偏离较大,仅边缘相关。引用 4 inbox
Michael D. Moffitt, Pratik Fegade
crossref: 2026-09-12dblp: 2026-09-12

AI 速览

一句话:回顾分布式深度学习算子内并行竞赛,发布基准并总结模型划分挑战与提交质量。

本文回顾了 ASPLOS 2025 / EuroSys 2025 分布式深度学习算子内并行竞赛。大规模深度学习依赖多加速器分布式计算,编译器需为模型操作选择合理并行策略;由于神经网络支持多种并行方式,为每条指令确定策略是困难的组合优化问题。为征集新思路,作者组织为期数月的竞赛,推动模型划分算法发展。文章介绍基本问题形式化、关键挑战与机会、新基准套件,并总结参赛提交质量与赛事经验,为后续自动并行与模型划分研究提供参考。

问题
在分布式深度学习中,编译器如何为模型操作自动选择合理的算子内并行策略;该问题是一个具有多种并行选择的困难组合优化问题。
方法
组织多个月竞赛,并给出问题形式化、关键挑战与机会、新基准套件,以征集和比较模型划分算法。
评估
通过赛事回顾,基于新基准套件和收到的参赛提交质量来总结当前模型划分算法的进展。
原文摘要(Abstract)

A chief enabler of large-scale deep learning is the distribution of computation across multiple interconnected hardware accelerators. In order to unlock the maximum possible performance, a compiler must first select a reasonable strategy to parallelize a model's operations. Since neural network architectures admit multiple flavors of parallelism, determining the proper strategy for each instruction is a critical (albeit non-trivial) task. To solicit new ideas toward solving this challenging combinatorial optimization problem, we organized the ASPLOS 2025 / EuroSys 2025 Contest on Intra-Operator Parallelism for Distributed Deep Learning, a multi-month competition focused on advancing the state-of-the-art for model partitioning algorithms. In this paper, we offer a retrospective of this event, including the basic problem formulation, key challenges & opportunities, our new benchmark suite, and the quality of submissions received.

链接与引用

DOI 原文 · PDF(开放获取) · DBLP

BibTeX
@inproceedings{MoffittF25,
  title = {The ASPLOS 2025 / EuroSys 2025 Contest on Intra-Operator Parallelism for Distributed Deep Learning},
  author = {Michael D. Moffitt and Pratik Fegade},
  booktitle = {ASPLOS},
  year = {2025},
  doi = {10.1145/3676642.3736399}
}