paperbot · PL 论文追踪

全部论文 待读 收藏

P2 Getting to the point: index sets and parallelism-preserving autodiff for pointful array programming

切入要点:pointful 数组编程的索引集与保持并行的自动微分
ICFP 5(ICFP)2021匹配度 3/10 · 论文属于编程语言设计、类型系统与数组/自动微分方向,虽然与 PL 及数值/ML 系统有泛化关联,但并未涉及用户核心关注的静态分析、LLM/Agentic 程序、浮点程序分析或贝叶斯程序分析。引用 39 inbox
Adam Paszke, Daniel D. Johnson, David Duvenaud, Dimitrios Vytiniotis, Alexey Radul, Matthew J. Johnson, Jonathan Ragan-Kelley, Dougal Maclaurin
crossref: 2026-09-10

AI 速览

一句话:提出将数组视为类型化索引集上的函数,并用类型化效应支持可自动并行化原地更新及保持并行的反向自动微分。

该论文针对高层函数式语言的清晰安全与底层数值语言的效率并行难以兼得的问题,提出一种新的数组编程语言设计。其核心是将数组视为类型化索引集上的即时记忆化函数,使柯里化等函数操作可直接作用于数组;并采用显式嵌套索引风格,以对应函数应用。语言引入细粒度类型化效应系统,支持简洁且自动并行化的原地更新;其中结合性累加效应使反向模式自动微分能够处理原地更新并保持并行性。实验以 Futhark 为基准,表明通过激进内联和类型驱动编译,点式风格的表达性数组程序性能损失很小。

问题
如何兼得高层函数式数组语言的清晰安全与底层数值语言的效率和并行性,尤其是让原地更新可自动并行化并支持保持并行的反向自动微分。
方法
将数组建模为类型化索引集上的即时记忆化函数,采用显式嵌套索引风格,并设计细粒度类型化效应系统(含结合性累加效应)来编译原地更新和反向自动微分。
评估
以 Futhark 数组语言为基准进行性能对比,显示激进内联与类型驱动编译能让点式数组程序在保持表达性的同时仅有很小性能损失。
原文摘要(Abstract)

We present a novel programming language design that attempts to combine the clarity and safety of high-level functional languages with the efficiency and parallelism of low-level numerical languages. We treat arrays as eagerly-memoized functions on typed index sets, allowing abstract function manipulations, such as currying, to work on arrays. In contrast to composing primitive bulk-array operations, we argue for an explicit nested indexing style that mirrors application of functions to arguments. We also introduce a fine-grained typed effects system which affords concise and automatically-parallelized in-place updates. Specifically, an associative accumulation effect allows reverse-mode automatic differentiation of in-place updates in a way that preserves parallelism. Empirically, we benchmark against the Futhark array programming language, and demonstrate that aggressive inlining and type-driven compilation allows array programs to be written in an expressive, "pointful" style with little performance penalty.

链接与引用

DOI 原文 · PDF(开放获取) ·

BibTeX
@article{paperbot1137,
  title = {Getting to the point: index sets and parallelism-preserving autodiff for pointful array programming},
  author = {Adam Paszke and Daniel D. Johnson and David Duvenaud and Dimitrios Vytiniotis and Alexey Radul and Matthew J. Johnson and Jonathan Ragan-Kelley and Dougal Maclaurin},
  journal = {Proceedings of the ACM on Programming Languages},
  volume = {5},
  number = {ICFP},
  year = {2021},
  doi = {10.1145/3473593}
}