一句话：提出 Jigsaw，用程序分析与合成后处理增强大语言模型，实现基于多模态意图的 Pandas API 代码合成。
大型预训练语言模型已能根据自然语言意图生成代码，但它们不理解程序语义，无法保证生成代码的质量。本文提出用程序分析与合成技术对这类模型的输出进行后处理，使其理解程序语法与语义，并能利用用户反馈在使用中持续改进。作者构建并评估了工具 Jigsaw，面向 Python Pandas API，支持多模态输入来合成代码。经验表明，随着大语言模型在意图到代码合成上的演进，Jigsaw 这类方法在提升系统准确性方面仍将发挥重要作用。
Large pre-trained language models such as GPT-3 [10], Codex [11], and Google's language model [7] are now capable of generating code from natural language specifications of programmer intent. We view these developments with a mixture of optimism and caution. On the optimistic side, such large language models have the potential to improve productivity by providing an automated AI pair programmer for every programmer in the world. On the cautionary side, since these large language models do not understand program semantics, they offer no guarantees about quality of the suggested code. In this paper, we present an approach to augment these large language models with post-processing steps based on program analysis and synthesis techniques, that understand the syntax and semantics of programs. Further, we show that such techniques can make use of user feedback and improve with usage. We present our experiences from building and evaluating such a tool Jigsaw, targeted at synthesizing code for using Python Pandas API using multi-modal inputs. Our experience suggests that as these large language models evolve for synthesizing code from intent, Jigsaw has an important role to play in improving the accuracy of the systems.
DOI 原文 ·
@inproceedings{paperbot8615,
title = {Jigsaw},
author = {Naman Jain and Skanda Vaidyanath and Arun Iyer and Nagarajan Natarajan and Suresh Parthasarathy and Sriram Rajamani and Rahul Sharma},
booktitle = {ICSE},
year = {2022},
doi = {10.1145/3510003.3510203}
}