paperbot · PL 论文追踪

全部论文 待读 收藏

P2 Backprop as Functor: A compositional perspective on supervised learning

反向传播作为函子:监督学习的组合视角
LICS 2019 34th Annual ACM/IEEE Symposium on Logic in Computer Science (LICS)2019匹配度 3/10 · 该论文从范畴论角度形式化反向传播与监督学习,属于机器学习与程序语言的交叉,但主题偏基础理论,并未涉及静态分析、Agentic/LLM、浮点程序分析或贝叶斯程序分析等用户核心方向,也没有工程落地或LLM结合。引用 12 inbox
Brendan Fong, David Spivak, Remy Tuyeras
crossref: 2026-09-12

AI 速览

一句话:将反向传播范畴化:证明梯度下降是从参数化函数到更新规则范畴的 monoidal functor,并引入 request function。

监督学习算法在由参数空间 P 参数化的函数集 A→B 中搜索,以逼近理想函数 f:A→B,并依据样例 (a,f(a)) 更新参数。本文定义一个更新规则可组合的范畴,把参数化函数视为对象/态射,更新规则视为可组合过程。核心贡献是 request function,用结构化方式描述反向传播:训练时先请求误差信息,再反向传播更新。作者证明,在固定步长和满足特定性质的误差函数下,梯度下降定义了一个从参数化函数范畴到更新规则范畴的 monoidal functor。该视角推广了神经网络,并把反向传播与双向编程和 open games 中的结构联系起来。

问题
如何用范畴论与组合视角统一理解监督学习中的参数更新与反向传播,并推广神经网络等模型。
方法
定义更新规则范畴与 request function,证明在固定步长和特定误差函数下,梯度下降构成从参数化函数范畴到更新规则范畴的 monoidal functor。
评估
以范畴论构造和定理证明为主要评估方式,并通过与双向编程、open games 的联系展示其泛化性与结构性解释;未报告实验评估。
原文摘要(Abstract)

A supervised learning algorithm searches over a set of functions$A\rightarrow B$parametrised by a space$P$to find the best approximation to some ideal function$f:A\rightarrow B$. It does this by taking examples$(a, f(a))\in A\times B$, and updating the parameter according to some rule. We define a category where these update rules may be composed, and show that gradient descent-with respect to a fixed step size and an error function satisfying a certain property-defines a monoidal functor from a category of parametrised functions to this category of update rules. A key contribution is the notion of request function. This provides a structural perspective on backpropagation, giving a broad generalisation of neural networks and linking it with structures from bidirectional programming and open games.

链接与引用

DOI 原文 · PDF(开放获取) ·

BibTeX
@inproceedings{paperbot4969,
  title = {Backprop as Functor: A compositional perspective on supervised learning},
  author = {Brendan Fong and David Spivak and Remy Tuyeras},
  booktitle = {LICS},
  year = {2019},
  doi = {10.1109/lics.2019.8785665}
}