一句话:提出将数组视为类型化索引集上的函数,并用类型化效应支持可自动并行化原地更新及保持并行的反向自动微分。
该论文针对高层函数式语言的清晰安全与底层数值语言的效率并行难以兼得的问题,提出一种新的数组编程语言设计。其核心是将数组视为类型化索引集上的即时记忆化函数,使柯里化等函数操作可直接作用于数组;并采用显式嵌套索引风格,以对应函数应用。语言引入细粒度类型化效应系统,支持简洁且自动并行化的原地更新;其中结合性累加效应使反向模式自动微分能够处理原地更新并保持并行性。实验以 Futhark 为基准,表明通过激进内联和类型驱动编译,点式风格的表达性数组程序性能损失很小。
We present a novel programming language design that attempts to combine the clarity and safety of high-level functional languages with the efficiency and parallelism of low-level numerical languages. We treat arrays as eagerly-memoized functions on typed index sets, allowing abstract function manipulations, such as currying, to work on arrays. In contrast to composing primitive bulk-array operations, we argue for an explicit nested indexing style that mirrors application of functions to arguments. We also introduce a fine-grained typed effects system which affords concise and automatically-parallelized in-place updates. Specifically, an associative accumulation effect allows reverse-mode automatic differentiation of in-place updates in a way that preserves parallelism. Empirically, we benchmark against the Futhark array programming language, and demonstrate that aggressive inlining and type-driven compilation allows array programs to be written in an expressive, "pointful" style with little performance penalty.
@article{paperbot1137,
title = {Getting to the point: index sets and parallelism-preserving autodiff for pointful array programming},
author = {Adam Paszke and Daniel D. Johnson and David Duvenaud and Dimitrios Vytiniotis and Alexey Radul and Matthew J. Johnson and Jonathan Ragan-Kelley and Dougal Maclaurin},
journal = {Proceedings of the ACM on Programming Languages},
volume = {5},
number = {ICFP},
year = {2021},
doi = {10.1145/3473593}
}