一句话：构建 Copilot 生成 Python 代码的代码坏味数据集，并评估 Copilot Chat 在不同提示下修复坏味的效果。
Python 中代码坏味会降低可读性与可维护性，而 GitHub Copilot 等大模型代码生成工具生成的代码也可能包含坏味。本文构建了包含 102 个代码坏味的 Copilot 生成 Python 代码数据集，首先探究坏味的出现情况，再评估 Copilot Chat 在不同提示（prompt）下修复这些坏味的效果。结果表明，10 类代码坏味中有 8 类可在 Copilot 生成的 Python 代码中检出，其中多层嵌套容器（Multiply-Nested Container）最为常见；Copilot Chat 的最高修复率达 87.1%，显示出修复自身生成代码坏味的潜力，并且提供更详细的提示可以进一步提升修复效果。
As one of the most popular dynamic languages, Python experiences a decrease in readability and maintainability when code smells are present. Recent advancements in Large Language Models have sparked growing interest in AI-enabled tools for both code generation and refactoring. GitHub Copilot is one such tool that has gained widespread usage. Copilot Chat, released in September 2023, functions as an interactive tool aimed at facilitating natural language-powered coding. However, limited attention has been given to understanding code smells in Copilot-generated Python code and Copilot Chat’s ability to fix the code smells. To this end, we built a dataset comprising 102 code smells in Copilot-generated Python code. Our aim is to first explore the occurrence of code smells in Copilot-generated Python code and then evaluate the effectiveness of Copilot Chat in fixing these code smells employing different prompts. The results show that 8 out of 10 types of code smells can be detected in Copilot-generated Python code, among which Multiply-Nested Container is the most common one. For these code smells, Copilot Chat achieves a highest fixing rate of 87.1%, showing promise in fixing Python code smells generated by Copilot itself. In addition, the effectiveness of Copilot Chat in fixing these smells can be improved by providing more detailed prompts.CCS CONCEPTS• Software and its engineering → Software maintenance tools.
@inproceedings{Zhang0FFL24,
title = {Copilot-in-the-Loop: Fixing Code Smells in Copilot-Generated Python Code using Copilot},
author = {Beiqi Zhang and Peng Liang and Qiong Feng and Yujia Fu and Zengyang Li},
booktitle = {ASE},
year = {2024},
doi = {10.1145/3691620.3695290}
}