一句话：把生成器的随机选择权重视为可学习参数，在离散概率编程系统中自动离线优化测试输入分布，显著提升找 bug 效率。
基于属性的测试（PBT）通过在随机生成的输入上执行可执行规约来验证软件。PBT 用户通常用生成器描述如何通过随机选择采样测试输入；为获得理想的输入分布，用户必须手动调优生成器，即决定各个随机选择的权重。然而，从单个权重推断最终分布极为困难，这一过程繁琐且限制了实际可达的分布。本文提出生成器的自动离线调优技术：给定带未定符号权重的生成器和目标函数，方法自动学习使目标最优的权重值。作者给出若干有用目标函数，支持（1）逼近期望分布、（2）提升测试用例的多样性与有效性。该方法实现在新型离散概率编程系统 Loaded Dice 中，该系统支持微分与参数学习，并被用作生成器语言。实验表明，该方法能按指定目标有效优化生成器分布；在 PBT 基准上的全面评估显示，当自动按多样性与有效性调优时，生成器的找 bug 速度提升 3.1–7.4 倍。
Property-based testing validates software against an executable specification by evaluating it on randomly generated inputs. The standard way that PBT users generate test inputs is via generators that describe how to sample test inputs through random choices. To achieve a good distribution over test inputs, users must tune their generators, i.e., decide on the weights of these individual random choices. Unfortunately, it is very difficult to understand how to choose individual generator weights in order to achieve a desired distribution, so today this process is tedious and limits the distributions that can be practically achieved. In this paper, we develop techniques for the automatic and offline tuning of generators. Given a generator with undetermined symbolic weights and an objective function, our approach automatically learns values for these weights that optimize for the objective. We describe useful objective functions that allow users to (1) target desired distributions and (2) improve the diversity and validity of their test cases. We have implemented our approach in a novel discrete probabilistic programming system, Loaded Dice , that supports differentiation and parameter learning, and use it as a language for generators. We empirically demonstrate that our approach is effective at optimizing generator distributions according to the specified objective functions. We also perform a thorough evaluation on PBT benchmarks, demonstrating that, when automatically tuned for diversity and validity, the generators exhibit a 3.1–7.4× speedup in bug finding.
@article{paperbot2948,
title = {Tuning Random Generators: Property-Based Testing as Probabilistic Programming},
author = {Ryan Tjoa and Poorva Garg and Harrison Goldstein and Todd Millstein and Benjamin C. Pierce and Guy Van den Broeck},
journal = {Proceedings of the ACM on Programming Languages},
volume = {9},
number = {OOPSLA2},
year = {2025},
doi = {10.1145/3763082}
}