一句话：提出用线性因子分解等常见函数式程序变换，将对偶数反向模式自动微分优化为复杂度正确且可在 Haskell 中高效实现的算法。
对偶数反向模式自动微分（AD）试图通过为每个标量值配对反向传播函数来实现反向 AD，但其在高阶输入语言上的正确性与效率此前仅基于自定义操作语义分析，能否高效实现尚不明确。本文受 Brunel 等人提出的线性因子分解技术启发，采用函数式编程社区中常见的程序变换（核心为线性因子分解）对双数反向 AD 进行优化，使其达到正确的复杂度，并可在支持可变数组的标准函数式语言（如 Haskell）中高效实现。除线性因子分解外，其余优化步骤均为已知的函数式编程思想。作者提供了一个实用实现，可对 Haskell98 的大部分语言特性进行微分，从而验证了该技术的可行性与效率。
Where dual-numbers forward-mode automatic differentiation (AD) pairs each scalar value with its tangent value, dual-numbers reverse-mode AD attempts to achieve reverse AD using a similarly simple idea: by pairing each scalar value with a backpropagator function. Its correctness and efficiency on higher-order input languages have been analysed by Brunel, Mazza and Pagani, but this analysis used a custom operational semantics for which it is unclear whether it can be implemented efficiently. We take inspiration from their use of linear factoring to optimise dual-numbers reverse-mode AD to an algorithm that has the correct complexity and enjoys an efficient implementation in a standard functional language with support for mutable arrays, such as Haskell. Aside from the linear factoring ingredient, our optimisation steps consist of well-known ideas from the functional programming community. We demonstrate the use of our technique by providing a practical implementation that differentiates most of Haskell98.
@article{paperbot2089,
title = {Efficient Dual-Numbers Reverse AD via Well-Known Program Transformations},
author = {Tom J. Smeding and Matthijs I. L. Vákár},
journal = {Proceedings of the ACM on Programming Languages},
volume = {7},
number = {POPL},
year = {2023},
doi = {10.1145/3571247}
}