一句话：提出即时捕捉测试生成，利用代码变更感知与 LLM 筛选生成会失败的测试，在合入前暴露缺陷并降低误报审查负担。
论文报告了 Meta 面向数亿行代码的大规模后端系统所设计的即时捕捉测试生成（Just-in-Time catching test generation）。与生成时即通过的传统加固测试不同，捕捉测试刻意失败，以便在代码合入前暴露缺陷，其核心挑战是避免误报失败带来的开发拖累。作者分析 22,126 个生成测试，发现代码变更感知方法在候选捕获生成上比加固测试提升 4 倍、比偶然失败测试提升 20 倍；为处理误报，引入基于规则与基于 LLM 的评估器，将人工审查负担降低 70%。推断统计分析显示，人工接受的变更被判定含显著更多误报，人工拒绝的变更含显著更多真阳性。最终向工程师上报 41 个候选捕获，8 个确认为真阳性，其中 4 个若未被发现将导致严重故障，表明该方法可扩展且能防止严重故障进入生产。
We report on Just-in-Time catching test generation at Meta, designed to prevent bugs in large scale backend systems of hundreds of millions of line of code. Unlike traditional hardening tests, which pass at generation time, catching tests are meant to fail, surfacing bugs before code lands. The primary challenge is to reduce development drag from false positive test failures. Analyzing 22,126 generated tests, we show code-change-aware methods improve candidate catch generation 4x over hardening tests and 20x over coincidentally failing tests. To address false positives, we use rule-based and LLM-based assessors. These assessors reduce human review load by 70%. Inferential statistical analysis showed that human-accepted code changes are assessed to have significantly more false positives, while human-rejected changes have significantly more true positives. We reported 41 candidate catches to engineers; 8 were confirmed to be true positives, 4 of which would have led to serious failures had they remained uncaught. Overall, our results show that Just-in-Time catching is scalable, industrially applicable, and that it prevents serious failures from reaching production.
DOI 原文 · arXiv · PDF（开放获取） · DBLP
@inproceedings{abs-2601-22832,
title = {Just-in-Time Catching Test Generation at Meta},
author = {Matthew Becker and Yifei Chen and Nicholas Cochran and Pouyan Ghasemi and Abhishek Gulati and Mark Harman and Zachary Haluza and Mehrdad Honarkhah and Herve Robert and Jiacheng Liu and Weini Liu and Sreeja Thummala and Xiaoning Yang and Rui Xin and Sophie Zeng},
booktitle = {FSE},
year = {2026},
doi = {10.1145/3803437.3805202}
}