一句话：本文评估 TODO 注释能否增强提示工程，以及 GitHub Copilot 会偿还还是重现自认技术债。
技术债是阻碍未来开发的次优方案或未解决问题，开发者常在源码注释中“自认”技术债（SATD）。尽管技术债维护成本高、代码智能工具进步快，此前尚无工作把 SATD 纳入提示工程。本文构建并分析来自 102,424 个仓库最新修订版的 36,381 条 TODO 注释数据集，并抽样人工使用 GitHub Copilot 生成 1,140 段代码。实验显示，Copilot 会在有提示和无提示时生成带有 SATD 症状的代码；同时，它在不同条件下也能自动偿还 SATD。作者进一步定性考察成功与失败注释的特征，并讨论未来工具和研究者在 AI 辅助软件维护上的改进空间。
Code intelligence tools such as GitHub Copilot have begun to bridge the gap between natural language and programming language. A frequent software development task is the management of technical debts, which are suboptimal solutions or unaddressed issues which hinder future software development. Developers have been found to "self-admit" technical debts (SATD) in software artifacts such as source code comments. Thus, is it possible that the information present in these comments can enhance code generative prompts to repay the described SATD? Or, does the inclusion of such comments instead cause code generative tools to reproduce the harmful symptoms of described technical debt? Does the modification of SATD impact this reaction? Despite the heavy maintenance costs caused by technical debt and the recent improvements of code intelligence tools, no prior works have sought to incorporate SATD towards prompt engineering. Inspired by this, this paper contributes and analyzes a dataset consisting of 36,381 TODO comments in the latest available revisions of their respective 102,424 repositories, from which we sample and manually generate 1,140 code bodies using GitHub Copilot. Our experiments show that GitHub Copilot can generate code with the symptoms of SATD, both prompted and unprompted. Moreover, we demonstrate the tool's ability to automatically repay SATD under different circumstances and qualitatively investigate the characteristics of successful and unsuccessful comments. Finally, we discuss gaps in which GitHub Copilot's successors and future researchers can improve upon code intelligence tasks to facilitate AI-assisted software maintenance.
@inproceedings{paperbot11337,
title = {Are Prompt Engineering and TODO Comments Friends or Foes? An Evaluation on GitHub Copilot},
author = {David OBrien and Sumon Biswas and Sayem Mohammad Imtiaz and Rabe Abdalkareem and Emad Shihab and Hridesh Rajan},
booktitle = {ICSE},
year = {2024},
doi = {10.1145/3597503.3639176}
}