一句话：提出概率程序平滑性静态分析，自动识别可微部分，改进路径梯度估计器且无需人工干预。
本文针对概率程序中路径梯度估计器仅适用于可微模型、非可微情形需人工处理的问题，提出一种静态分析，用于发现程序的可微或更一般的平滑部分。由于部分可微性、部分连续性等平滑性质不被函数复合保持，顺序复合的可靠分析极具挑战；作者对目标平滑性质提出五条假设，并在假设下证明分析可靠性，且主要例子满足这些假设。基于可微性实例化，改进的梯度估计器对平滑部分使用路径法，其余部分使用得分估计器，并在正则条件下保持无偏。在 Pyro 代表性程序上的实验表明，该分析能准确识别平滑部分，并充分释放改进估计器的性能。
We present a static analysis for discovering differentiable or more generally smooth parts of a given probabilistic program, and show how the analysis can be used to improve the pathwise gradient estimator, one of the most popular methods for posterior inference and model learning. Our improvement increases the scope of the estimator from differentiable models to non-differentiable ones without requiring manual intervention of the user; the improved estimator automatically identifies differentiable parts of a given probabilistic program using our static analysis, and applies the pathwise gradient estimator to the identified parts while using a more general but less efficient estimator, called score estimator, for the rest of the program. Our analysis has a surprisingly subtle soundness argument, partly due to the misbehaviours of some target smoothness properties when viewed from the perspective of program analysis designers. For instance, some smoothness properties, such as partial differentiability and partial continuity, are not preserved by function composition, and this makes it difficult to analyse sequential composition soundly without heavily sacrificing precision. We formulate five assumptions on a target smoothness property, prove the soundness of our analysis under those assumptions, and show that our leading examples satisfy these assumptions. We also show that by using information from our analysis instantiated for differentiability, our improved gradient estimator satisfies an important differentiability requirement and thus computes the correct estimate on average (i.e., returns an unbiased estimate) under a regularity condition. Our experiments with representative probabilistic programs in the Pyro language show that our static analysis is capable of identifying smooth parts of those programs accurately, and making our improved pathwise gradient estimator exploit all the opportunities for high performance in those programs.
@article{paperbot2067,
title = {Smoothness Analysis for Probabilistic Programs with Application to Optimised Variational Inference},
author = {Wonyeol Lee and Xavier Rival and Hongseok Yang},
journal = {Proceedings of the ACM on Programming Languages},
volume = {7},
number = {POPL},
year = {2023},
doi = {10.1145/3571205}
}