一句话：构建源自GitHub真实开发历史的AIGC检测基准，支持多粒度检测，并对28种方法进行实证评估。
现有AIGC检测基准与真实开发流程脱节。本文构建RealAIGC基准，数据取自GitHub真实仓库开发历史，包含五种编码代理生成的PR与人类代码，覆盖八种主流语言，支持仓库、PR、提交、文件及函数五级粒度检测，含超四百万个函数。基于此，作者对三类共28种检测方法进行实证评估。结果显示：监督方法仅在同质单语言小型仓库中表现良好，在主流动态语言和大规模多开发者环境中显著退化；基于困惑度的方法几乎无效；没有方法具备通用适用性，且在提交与PR级别均表现不佳。该工作为推进AIGC检测提供实证指导。
AI-generated code (AIGC) from autonomous coding agents are widely adopted across the software development lifecycle. However, AIGC also introduces critical challenges to open-source community governance, making AIGC detection important. Existing AIGC detection benchmarks are fundamentally misaligned with real-world development processes. To address this gap, we construct RealAIGC, a benchmark for AIGC detection grounded in real-world code repositories. RealAIGC is built from the development histories of GitHub and includes pull request (PR) generated by five coding agents alongside human-written code, covering eight major programming languages. It supports multi-granularity detection at the repository, PR, commit, file, and function levels, with more than four million functions. Based on RealAIGC, we conduct a realistic empirical evaluation of 28 AIGC detection methods across three categories. The results show that supervised approaches perform well only in small, homogeneous, single-language repositories, but degrade significantly in mainstream dynamic languages and in large-scale, multi-developer environments. Perplexity based methods are nearly ineffective in realistic settings. No existing method demonstrates general applicability, and all approaches perform unsatisfactorily at the commit and PR levels. This work provides empirical guidance for advancing AIGC detection techniques.
@inproceedings{paperbot12288,
title = {RealAIGC: Towards More Realistic Evaluation of AI-Generated Code Detection in Real-World Code Repository},
author = {Wentao Chen and Yufei Chen and Huiqun Yu and Guisheng Fan and Kaiwen Zhi and Peite Guo and Jianyuan Yu},
booktitle = {ASE},
year = {2026},
doi = {10.1145/3832783.3834411}
}