paperbot · PL 论文追踪

全部论文 待读 收藏

P2 Towards automating code review at scale

迈向大规模代码评审自动化
FSE Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering2021匹配度 3/10 · 论文聚焦大规模代码评审自动化,属于软件工程与机器学习交叉,而非用户核心的静态分析、形式化验证、浮点或贝叶斯程序分析,与PL与AI的双向结合点关联较弱;但其问题驱动地揭示任务困难并收集大规模数据集,对代码相关ML任务有间接参考价值。引用 23 inbox
Vincent J. Hellendoorn, Jason Tsay, Manisha Mukherjee, Martin Hirzel
crossref: 2026-09-12

AI 速览

一句话:本文研究大规模代码评审自动化的愿景与挑战,通过收集数十万个changeset并预测评论位置,揭示该任务的困难与随机性。

神经方法日益用于支持与自动化软件开发任务,代码评审是自然的下一目标。但基于历史评审训练模型模仿开发者并不简单:开源评审在质量、措辞与深度上因评审者而异,且 changeset 常达数千 token,超出当前神经模型容量;已有工作通过聚焦小输入且已知评论位置来回避问题。本文考察真实规模下自动化代码评审的愿景与挑战:从数百个常规进行评审的项目收集数十万个 changeset,聚焦于预测评论位置这一稀疏任务。通过分析模型性能与数据集统计,作者表明即使该任务也已颇具挑战,很大程度上源于评审中巨大的变化与(表面上的)随机性,并由此引出研究议程。

问题
在真实规模下自动化代码评审是否可行?当前神经方法在预测代码评审评论位置这一稀疏任务上表现如何?现有工作往往通过聚焦小输入和已知评论发生位置来回避大规模 changeset 的挑战。
方法
作者从数百个常规进行代码评审的项目中收集数十万个 changeset(许多包含数千 token),并训练/评估模型来预测评论出现的位置,同时分析数据集统计与模型性能。
评估
通过分析模型在预测评论位置上的性能以及数据集统计,揭示该任务即使在理想化设定下也极具挑战,主要源于代码评审中巨大的变化与(表面上的)随机性。
原文摘要(Abstract)

As neural methods are increasingly used to support and automate software development tasks, code review is a natural next target. Yet, training models to imitate developers based on past code reviews is far from straightforward: reviews found in open-source projects vary greatly in quality, phrasing, and depth depending on the reviewer. In addition, changesets are often large, stretching the capacity of current neural models. Recent work reported modest success at predicting review resolutions, but largely side-stepped the above issues by focusing on small inputs where comments were already known to occur. This work examines the vision and challenges of automating code review at realistic scale. We collect hundreds of thousands of changesets across hundreds of projects that routinely conduct code review, many of which change thousands of tokens. We focus on predicting just the locations of comments, which are quite rare. By analyzing model performance and dataset statistics, we show that even this task is already challenging, in no small part because of tremendous variation and (apparent) randomness in code reviews. Our findings give rise to a research agenda for realistically and impactfully automating code review.

链接与引用

DOI 原文 · PDF(开放获取) ·

BibTeX
@inproceedings{paperbot6306,
  title = {Towards automating code review at scale},
  author = {Vincent J. Hellendoorn and Jason Tsay and Manisha Mukherjee and Martin Hirzel},
  booktitle = {FSE},
  year = {2021},
  doi = {10.1145/3468264.3473134}
}