一句话：提出基于代码大模型Codex的自动程序修复系统PyDex，修复入门Python作业的语法与语义错误。
学生入门编程作业常出错，教师定制修复耗时。自动程序修复可合成修复，但已有符号与神经方法需大量工程或数据训练。本文提出PyDex，利用在代码上训练的大语言模型（Codex）构建面向入门Python作业的APR系统。它通过多模态提示、迭代查询、基于测试用例的few-shot选择以及程序分块，同时修复语法和语义错误。在286个真实学生程序上评估，并与三个基线（包括BIFI与Refactory的组合）比较，PyDex能修复更多程序且平均补丁更小。
Students often make mistakes in their introductory programming assignments as part of their learning process. Unfortunately, providing custom repairs for these mistakes can require a substantial amount of time and effort from class instructors. Automated program repair (APR) techniques can be used to synthesize such fixes. Prior work has explored the use of symbolic and neural techniques for APR in the education domain. Both types of approaches require either substantial engineering efforts or large amounts of data and training. We propose to use a large language model trained on code, such as Codex (a version of GPT), to build an APR system -- PyDex -- for introductory Python programming assignments. Our system can fix both syntactic and semantic mistakes by combining multi-modal prompts, iterative querying, test-case-based selection of few-shots, and program chunking. We evaluate PyDex on 286 real student programs and compare to three baselines, including one that combines a state-of-the-art Python syntax repair engine, BIFI, and a state-of-the-art Python semantic repair engine for student assignments, Refactory. We find that PyDex can fix more programs and produce smaller patches on average.
@article{paperbot2544,
title = {PyDex: Repairing Bugs in Introductory Python Assignments using LLMs},
author = {Jialu Zhang and José Pablo Cambronero and Sumit Gulwani and Vu Le and Ruzica Piskac and Gustavo Soares and Gust Verbruggen},
journal = {Proceedings of the ACM on Programming Languages},
volume = {8},
number = {OOPSLA1},
year = {2024},
doi = {10.1145/3649850}
}