一句话:提出基于代码语言模型的代码审查工具,检测 Python 变量误用并生成修复建议。
本文报告了 Code Understanding Linter 的开发经验,这是一种基于代码语言模型的自动代码审查工具,用于检测 Python 代码中的变量误用缺陷并建议修复方式。为提升实用性,作者提出组合两种不同语言模型、过滤模型的无意义输出,以及将模型输出解释为开发者友好的诊断信息。工具在三星电子包含真实 Python 代码的代码仓库上评估。实验表明该工具能发现真实代码中隐藏的缺陷,但误报率远高于预期。作者人工调查所有误报,讨论了语言模型的局限性及可能的解决方向。
We share our experience in developing Code Understanding Linter, an automated code review tool based on language models of code. We introduce several ideas to make the tool be more practical, including combining two different language models, filtering meaningless outputs from the model, and generating developer-friendly diagnosis messages by interpreting the outputs from the model. On top of those ideas, we describe the design and implementation of an automated code review tool to detect variable-misuse defects in Python codes and suggest how to fix them. We evaluated the tool with a set of code repositories in Samsung Electronics, which contains real-world Python codes. Our experiment proves that our tool can discover hidden defects in the real-world codes, but the false positive rate is far higher than we expected. After manually investigating every false positives, we discuss the limitations of the language models and possible solutions.