一句话：用范畴论中的延迟迹统一建模因果计算，并给出无需展开的状态网络反向传播微分算子。
论文研究因果计算：输出序列的第 n 项只依赖前 n 项输入。作者在范畴论中构造从笛卡尔范畴 C 到 St(C) 的转换，并引入一种新型的类迹运算——延迟迹。该运算不满足通常迹的 yanking 与 dinaturality 公理，却为 St(C) 提供带隐式受保护保证的反馈机制。当 C 具有笛卡尔微分算子时，作者借助时间反向传播（BPTT）的抽象版本，为 St(C) 构造微分算子，从而获得链式法则、Schwartz 定理等性质。该算子无需展开网络即可计算有状态网络的导数。
We investigate causal computations, which take sequences of inputs to sequences of outputs such that the nth output depends on the first n inputs only. We model these in category theory via a construction taking a Cartesian category \mathbbC to another category St(\mathbbC) with a novel trace-like operation called “delayed trace”, which misses yanking and dinaturality axioms of the usual trace. The delayed trace operation provides a feedback mechanism in St(\mathbbC) with an implicit guardedness guarantee. When \mathbbC is equipped with a Cartesian differential operator, we construct a differential operator for St (\mathbbC) using an abstract version of backpropagation through time, a technique from machine learning based on unrolling of functions. This obtains a swath of properties for backpropagation through time, including a chain rule and Schwartz theorem. Our differential operator is also able to compute the derivative of a stateful network without requiring the network to be unrolled.
DOI 原文 ·
@inproceedings{paperbot5023,
title = {Differentiable Causal Computations via Delayed Trace},
author = {David Sprunger and Shin-ya Katsumata},
booktitle = {LICS},
year = {2019},
doi = {10.1109/lics.2019.8785670}
}