paperbot · PL 论文追踪

全部论文 待读 收藏

P1 Proofs Promptly: Proof-Oriented Programming with AI Agents (Experience Report)

Proofs Promptly:用 AI Agent 进行面向证明的编程(经验报告)
ICFP 10(ICFP)2026匹配度 7/10 · 该论文直接对应研究画像中的“AI→PL”方向,即用 LLM/Agent 改进形式化验证与证明工程,且涉及 Agentic 程序;但它是经验报告、以案例和轶事为主,缺乏理论形式化或系统 benchmark,也不涉及静态分析、浮点程序分析或贝叶斯程序分析。引用 1 inbox
Eleftherios Ioannidis, Nikhil Swamy, Gabriel Ebner, Matthai Philipose, Tahina Ramananandro
crossref: 2026-09-10

AI 速览

一句话:经验报告:配备证明助手与 CLI 的 AI Agent 可自动产出数千行机器检验证明代码,把人类专家的工作压缩到给定问题描述与审查规约。

论文关注的问题是:AI 辅助编码的普及带来了更多软件缺陷,那么 AI 辅助的形式验证能否以同等规模减少缺陷?作者以经验报告的形式,记录了他们为 AI Agent 配备命令行接口与证明助手(如 F*/Lean 一类的交互式证明工具),在真实证明工程任务中产出数千行机器检验代码的轶事性经历。任务覆盖三类:为标准库实现经过验证的数据结构、把未验证代码翻译为形式语言并同时推断其规约、以及将既有证明移植并重构到新框架。作者总结了使「智能体式面向证明编程(agentic PoP)」有效或失效的具体技术,并刻画了人类专家的角色——其贡献缩减为提供自然语言的问题描述、审查自动生成的规约、偶尔补充一个关键不变式。结果显示这种人机分工为回路中的人类专家带来可观的杠杆效应:三位专家在两周内完成的案例研究,其手工证明工程成本估计约为半年。

问题
AI 辅助编码大规模普及的同时带来了更多软件缺陷,本文追问 AI 辅助形式验证能否以相近的规模与效率来消除缺陷,并厘清人在这一过程中应扮演什么角色。
方法
以经验报告形式,为配备 CLI 与证明助手的 AI Agent 设计智能体式面向证明编程(PoP)工作流,并在三类任务上实践:实现经验证的标准库数据结构、将未验证代码翻译为形式语言并推断规约、移植与重构既有证明。
评估
通过多个案例研究进行定性的经验评估,总结有效与无效的技术,并将三人两周的产出与估计约半年的手工证明工程成本进行对比。
原文摘要(Abstract)

The widespread adoption of AI-assisted coding is directly proportional to an increase in software bugs; can AI-assisted formal verification help reduce bugs at a comparable scale? In this experience report we give an anecdotal account of AI agents, equipped with a CLI and a proof assistant, producing thousands of lines of machine-checked code. We detail our experience across different proof-engineering tasks: implementing verified data structures for a standard library, translating unverified code into a formal language while inferring its specification, and porting and refactoring existing proofs to new frameworks. We present the techniques that made agentic proof-oriented programming (PoP) effective---or ineffective---and characterize the role of the human expert, whose contribution reduces to providing natural-language problem descriptions, reviewing auto-generated specifications, and occasionally supplying a key invariant. Our findings suggest that this division of labor provides substantial leverage to the human expert in the loop: three experts, over the course of two weeks, completed case studies whose manual proof-engineering cost we estimate at roughly half a year.

链接与引用

DOI 原文 · PDF(开放获取) · DBLP

BibTeX
@article{IoannidisSEPR26,
  title = {Proofs Promptly: Proof-Oriented Programming with AI Agents (Experience Report)},
  author = {Eleftherios Ioannidis and Nikhil Swamy and Gabriel Ebner and Matthai Philipose and Tahina Ramananandro},
  journal = {Proceedings of the ACM on Programming Languages},
  volume = {10},
  number = {ICFP},
  year = {2026},
  doi = {10.1145/3828709}
}