paperbot · PL 论文追踪

全部论文 待读 收藏

P2 Efficient CHAD

高效 CHAD
POPL 8(POPL)2024匹配度 3/10 · 论文属于 PL 与 ML 交叉的自动微分/函数式程序变换,含 Agda 复杂度证明和工程优化,符合对理论形式化与实现的偏好;但它不涉及静态分析、Agentic/LLM、浮点程序分析或贝叶斯程序分析等用户核心方向。引用 6 inbox
Tom J. Smeding, Matthijs I. L. Vákár
crossref: 2026-09-10

AI 速览

一句话:用稀疏向量、状态传递和去函数化优化 CHAD,使反向模式自动微分达到预期复杂度,并给出 Agda 复杂度证明。

本文研究如何让基本组合同态自动微分(CHAD)在保持简单、可组合和通用性的同时,达到反向模式自动微分应有的计算复杂度。作者组合稀疏向量、状态传递风格代码以及高阶语言中的去函数化/闭包转换等已知优化,得到纯函数式算法,并用(函数式)可变更新消除最后的对数因子。作者在 Agda 中形式化了复杂度证明。论文还讨论将该技术用于并行函数式数组程序:所需可变性均为可交换、可结合的累加,从而保留任务并行性,并且可为大多数数据并行数组原语写出数据并行导数。

问题
基本 CHAD 算法虽简单通用,但直接实现无法达到反向模式自动微分应有的计算复杂度。
方法
通过稀疏向量、状态传递风格、去函数化/闭包转换及函数式可变更新等优化,构造纯函数式反向模式 AD 算法,并在 Agda 中形式化复杂度证明;同时分析其在并行数组程序上的应用。
评估
以复杂度证明和 Agda 机器化形式化评估算法复杂度,并讨论其在任务/数据并行函数式数组程序上的适用性。
原文摘要(Abstract)

We show how the basic Combinatory Homomorphic Automatic Differentiation (CHAD) algorithm can be optimised, using well-known methods, to yield a simple, composable, and generally applicable reverse-mode automatic differentiation (AD) technique that has the correct computational complexity that we would expect of reverse-mode AD. Specifically, we show that the standard optimisations of sparse vectors and state-passing style code (as well as defunctionalisation/closure conversion, for higher-order languages) give us a purely functional algorithm that is most of the way to the correct complexity, with (functional) mutable updates taking care of the final log-factors. We provide an Agda formalisation of our complexity proof. Finally, we discuss how the techniques apply to differentiating parallel functional array programs: the key observations are 1) that all required mutability is (commutative, associative) accumulation, which lets us preserve task-parallelism and 2) that we can write down data-parallel derivatives for most data-parallel array primitives.

链接与引用

DOI 原文 · PDF(开放获取) ·

BibTeX
@article{paperbot2629,
  title = {Efficient CHAD},
  author = {Tom J. Smeding and Matthijs I. L. Vákár},
  journal = {Proceedings of the ACM on Programming Languages},
  volume = {8},
  number = {POPL},
  year = {2024},
  doi = {10.1145/3632878}
}