一句话：提出两阶段 LLM 代码审查框架 BitsAI-CR，以 RuleChecker 检测、ReviewFilter 校验，并在字节跳动大规模落地。
代码审查在大型工业环境中耗费大量人力，现有 LLM 方案在精度与实用性上仍显不足。本文提出 BitsAI-CR，采用两阶段框架：RuleChecker 基于审查规则分类体系初步发现缺陷，ReviewFilter 再做精度校验；同时设计数据飞轮，通过结构化反馈与评估指标持续迭代。作者提出 Outdated Rate 指标，用以反映开发者对审查意见的实际采纳情况，从而支持规模化自动评估与系统优化。评估显示其审查评论生成精确率达 75.0%；针对字节跳动主要使用的 Go 语言，Outdated Rate 为 26.7%。系统已在字节跳动部署，周活跃用户超过 12000 人。
Code review remains a critical yet resource-intensive process in software development, particularly challenging in large-scale industrial environments. While Large Language Models (LLMs) show promise for automating code review, existing solutions face significant limitations in precision and practicality. This paper presents BitsAI-CR, an innovative framework that enhances code review through a two-stage approach combining RuleChecker for initial issue detection and ReviewFilter for precision verification. The system is built upon a comprehensive taxonomy of review rules and implements a data flywheel mechanism that enables continuous performance improvement through structured feedback and evaluation metrics. Our approach introduces an Outdated Rate metric that can reflect developers' actual adoption of review comments, enabling automated evaluation and systematic optimization at scale. Empirical evaluation demonstrates BitsAI-CR's effectiveness, achieving 75.0% precision in review comment generation. For the Go language which has predominant usage at ByteDance, we maintain an Outdated Rate of 26.7%. The system has been successfully deployed at ByteDance, serving over 12,000 Weekly Active Users (WAU). Our work provides valuable insights into the practical application of automated code review and offers a blueprint for organizations seeking to implement automated code reviews at scale.
@inproceedings{paperbot9065,
title = {BitsAI-CR: Automated Code Review via LLM in Practice},
author = {Tao Sun and Jian Xu and Yuanpeng Li and Zhao Yan and Ge Zhang and Lintao Xie and Lu Geng and Zheng Wang and Yueyan Chen and Qin Lin and Wenbo Duan and Kaixin Sui and Yuanshuo Zhu},
booktitle = {FSE},
year = {2025},
doi = {10.1145/3696630.3728552}
}