paperbot · PL 论文追踪

全部论文 待读 收藏

P2 Testing Theorems, Fully Automatically

全自动测试定理
OOPSLA 10(OOPSLA2)2026匹配度 3/10 · 论文属于形式化定理证明的自动化测试(Rocq/QuickChick),与用户关注的“AI→PL:利用LLM/Agent改进静态分析/形式化验证”方向仅在形式化验证大范畴上有交集,但完全未涉及LLM/Agent,也不属于静态分析、浮点程序分析或贝叶斯程序分析。引用 0 inbox
Segev Elazar Mittelman, Harrison Goldstein, Leonidas Lampropoulos
crossref: 2026-10-02

AI 速览

一句话:提出一种全自动、鲁棒于等价表述的定理性质测试策略,把生成器搜索建模为约束调度问题。

交互式定理证明中,先用基于性质的测试检验定理,有助于在投入证明前发现错误定义或错误陈述。但现代证明助手的测试基础设施仍不完善:Rocq 的 QuickChick 虽能为归纳关系随机生成满足约束的数据,却常需大量专业技巧,且效果高度依赖定理中假设顺序和归纳关系中约束顺序。本文提出一种全新定理测试策略,将 QuickChick 风格的性质与生成器空间刻画为约束调度问题,并引入归纳关系的“密度”概念估计生成器在任意输入下成功的倾向,从而自动搜索最佳性质或生成器。作者在 Rocq 的 QuickChick 上实现该算法,并用文献案例评估,表明一键自动化可与专家手工测试媲美,有时更能发现缺陷。

问题
在交互式定理证明中,QuickChick 等性质测试可帮助用户提前发现错误定义或定理陈述,但其有效使用依赖专家经验,并受假设与归纳约束顺序影响。
方法
将 QuickChick 式性质与生成器空间建模为约束调度问题,用归纳关系密度估计生成器有效性,并自动搜索最优方案。
评估
在 Rocq 的 QuickChick 上实现,通过多个文献案例研究评估,自动生成测试的效果与专家手工测试相当,部分情况下更能发现 bug。
原文摘要(Abstract)

While the ultimate goal of interactive theorem proving is to prove theorems, it can really help to test them first. Testing theorems, specifically using property-based testing, helps users identify incorrect definitions and theorem statements before they waste time on a proof that could never succeed. Unfortunately, the testing infrastructure provided by modern theorem provers has yet to reach its full potential. Even QuickChick, the state-of-the-art property-based testing framework for Rocq, which offers random generation for data satisfying inductively defined relations, often requires substantial effort and expertise to be used effectively. This is in part because this effectiveness is heavily sensitive to both the order that hypotheses appear within a theorem, and to the order that inductive constraints appear within the inductive relations involved. In this paper, we present a novel strategy for testing theorems that is highly effective, fully automatic, and robust to equivalent formulations of theorem and definition statements. To do so, we characterize the exponentially large space of possible QuickChick-style properties and generators as solutions to a constrained scheduling problem. To find the best property or generator in this space, we estimate effectiveness by introducing a notion of "density" for inductive relations, which approximates the tendency for a generator to succeed given arbitrary inputs. We implement our algorithm on top of the QuickChick framework for Rocq and evaluate it in a number of case studies from the literature, demonstrating that our push-button automation is on par with and in some cases even more effective at finding bugs than expertly handcrafted tests.

链接与引用

DOI 原文 · PDF(开放获取) ·

BibTeX
@article{paperbot12148,
  title = {Testing Theorems, Fully Automatically},
  author = {Segev Elazar Mittelman and Harrison Goldstein and Leonidas Lampropoulos},
  journal = {Proceedings of the ACM on Programming Languages},
  volume = {10},
  number = {OOPSLA2},
  year = {2026},
  doi = {10.1145/3839453}
}