paperbot · PL 论文追踪

全部论文 待读 收藏

P2 Formal Autograding in a Classroom (Extended Version)

课堂中的形式化自动评分(扩展版)
TOPLAS 48(3)2026匹配度 2/10 · 该论文是程序验证在本科编程课自动评分中的教育应用,不涉及用户核心关注的 LLM/Agent 与静态分析/形式化验证交叉、ML/Agentic 程序分析、浮点程序分析或贝叶斯程序分析。虽然属于形式化验证的应用,但与研究方向仅有非常边缘的关联。引用 0 inbox
Dragana Milovančević, Samuel Chassot, Mario Bucev, Marcin Wojnarowski, Viktor Kunčak
crossref: 2026-09-10

AI 速览

一句话:将程序验证器用于本科编程课自动评分,通过等价性检查评估学生提交并区分递归结构。

本文报告在本科编程课程中利用形式验证增强自动评分的经验。传统测试评分难以确认解法正确性,也难以区分输入输出行为相同但结构不同的提交。作者在原有测试基础设施上部署程序验证器,将学生提交与参考解进行等价性检查,并分析 11 个练习的 1700 余份提交。结果显示,评分器能证明提交正确并给出反例;等价性检查还可识别高层程序结构,尤其是递归模式,即使行为相同。由此提高对惯用解法正确性的信心,并更全面刻画解法空间,发现教师未预见的解法。

问题
如何提高本科编程课自动评分的可信度,并区分输入输出行为相同但高层结构不同的学生解法。
方法
在测试评分基础上接入程序验证器,检查学生提交与参考解之间的等价性,并利用等价性检查结果分析递归模式等程序结构。
评估
对 11 个编程练习的 1700 余份学生提交进行实验,表明评分器能证明正确性、报告反例,并揭示超越教师预期的解法空间。
原文摘要(Abstract)

We report our experience in enhancing automated grading in an undergraduate programming course using formal verification. In our experiments, we deploy a program verifier to check the equivalence between student submissions and our reference solutions, alongside the existing testing-based grading infrastructure. We collect and analyse over 1,700 student submissions to 11 programming exercises and show how our grader can prove submission correctness and report counterexamples. Beyond functional correctness, we were able to use the outcome of equivalence checks to differentiate student submissions according to their high-level program structure, in particular their recursion pattern, even when their input–output behaviour is identical. Consequently, we achieve (1) higher confidence in correctness of idiomatic solutions but also (2) more thorough assessment of solution landscape that reveals solutions beyond those envisioned by instructors.

链接与引用

DOI 原文 · PDF(开放获取) ·

BibTeX
@article{paperbot3944,
  title = {Formal Autograding in a Classroom (Extended Version)},
  author = {Dragana Milovančević and Samuel Chassot and Mario Bucev and Marcin Wojnarowski and Viktor Kunčak},
  journal = {ACM Transactions on Programming Languages and Systems},
  volume = {48},
  number = {3},
  year = {2026},
  doi = {10.1145/3828727}
}