paperbot · PL 论文追踪

全部论文 待读 收藏

P0 Planning to Hammer: Difficulty-Aware Decomposition for Automating Rocq Proofs

Planning to Hammer:面向 Rocq 证明自动化的难度感知分解
OOPSLA 10(OOPSLA2)2026匹配度 8/10 · 论文属于“AI→PL”方向,用 LLM/Agent 规划结合 CoqHammer 符号执行来改进 Rocq 定理证明自动化,符合用户对“LLM/Agent 改进形式化验证”的兴趣,且有难度模型、SerAPI/CoqHammer 工程实现和多 benchmark 评估。但它不涉及静态分析、浮点程序分析或贝叶斯程序分析,因此不是正中核心。引用 4 inbox
Ning Zhang, Nongyu Di, Zenan Li, Yuan Yao, Xiaoxing Ma
crossref: 2026-10-02

AI 速览

一句话:Quarry 将证明规划与执行分离,用 LLM 提出分解并由难度模型排序,递归调用 CoqHammer 提升 Rocq 证明自动化。

交互式定理证明器中的机器检查证明仍是软件形式验证的瓶颈:LLM 能提出高层证明策略但缺乏局部严谨性,CoqHammer 等自动策略能可靠解决局部目标却缺乏长程规划。为此,论文提出 Quarry,一个基于规划的证明合成框架,将证明规划与证明执行分离。Quarry 让 LLM 主动提出多种带任意子引理的证明分解,在 Rocq 中临时 admit 子引理并进行类型检查,再用基于证明状态的难度模型估计各候选的 hammer 可解性并排序;随后在有限预算内递归证明子引理,从而把长证明转化为一系列 hammer 可解的义务。系统实现于 SerAPI 和 CoqHammer 之上,并使用多个前沿 LLM 在多个基准上评估。在统一 10 分钟墙钟预算下,Quarry 在三个 Rocq 基准上比最强基线成功率提升 7–13 个百分点,表明神经规划与符号执行协同可实现可靠且成本可预测的证明自动化。

问题
在 Rocq 等交互式定理证明器中自动生成机器检查证明仍很困难,LLM 缺乏局部严谨性,而 CoqHammer 等自动策略缺乏长程规划能力。
方法
Quarry 让 LLM 提出多种带子引理的证明分解,临时 admit 子引理后在 Rocq 中类型检查,并用基于证明状态的难度模型估计 hammer 可解性以排序候选,再在预算内递归证明子引理。
评估
在 SerAPI 和 CoqHammer 上实现 Quarry,使用多个前沿 LLM 在多个 Rocq 基准上评估;在统一 10 分钟预算下,成功率比最强基线提升 7–13 个百分点。
原文摘要(Abstract)

As AI-generated code proliferates, formal verification—particularly through interactive theorem provers such as Rocq and Isabelle—becomes increasingly important for ensuring software correctness. However, producing machine-checked proofs in such provers remains a bottleneck. Existing solutions bring complementary strengths to proof automation: large language models (LLMs) can propose high-level proof strategies but lack local rigor; automated tactics such as CoqHammer can reliably discharge many local goals, but lack long-range planning capabilities. To combine the best of both worlds, we present Quarry, a planning-based proof synthesis framework that separates proof planning from proof execution. Specifically, Quarry asks an LLM to actively propose multiple proof decompositions with arbitrary sublemmas, type-checks them in Rocq under temporarily admitted sublemmas, and ranks candidates using a proof-state-based difficulty model estimating hammer solvability. It then recursively proves sublemmas within a bounded budget, effectively turning long proofs into sequences of hammer-solvable obligations. We implement Quarry on top of SerAPI and CoqHammer and evaluate it using multiple frontier LLMs across multiple benchmarks. The experimental results show that planning-based decomposition with solvability-aware ranking substantially improves automation while maintaining predictable cost. Under a uniform 10-minute wall-clock budget, Quarry improves over the strongest baseline by 7–13 percentage points in success rate across three Rocq benchmarks. These results demonstrate that reliable proof automation can be achieved by coordinating neural planning with symbolic execution rather than replacing either.

链接与引用

DOI 原文 · arXiv · PDF(开放获取) · DBLP

BibTeX
@article{abs-2606-17981,
  title = {Planning to Hammer: Difficulty-Aware Decomposition for Automating Rocq Proofs},
  author = {Ning Zhang and Nongyu Di and Zenan Li and Yuan Yao and Xiaoxing Ma},
  journal = {Proceedings of the ACM on Programming Languages},
  volume = {10},
  number = {OOPSLA2},
  year = {2026},
  doi = {10.1145/3839497}
}