一句话：提出 DynamicBatching 与 TestCaseBatching 两种测试批处理策略，在维持反馈时间的同时大幅减少 CI 所需测试机器。
大规模持续集成成本高昂，测试选择与优先级等优化技术旨在降低开销，而测试批处理这一有效手段却常被忽视。本文以 TestAll 为基线，研究并行度与机器数量对反馈时间的影响，重新评估 ConstantBatching，并提出两种新方法：DynamicBatching 依据队列中剩余变更动态调整批大小，TestCaseBatching 允许新构建在完整测试执行前加入批次。评估结合 Ericsson 的实践结果与 Chrome 开源项目的 2.76 亿条测试结果，衡量反馈时间与执行缩减量，并公开了 Chrome 项目脚本与数据。结果显示，测试并行化对反馈时间呈非线性影响，每次测试延迟会在整个测试队列中累积。批大小为 4 的 ConstantBatching 最多减少 72% 的机器并保持平均反馈时间，执行量恒定减少至多 75%；DynamicBatching 最多减少 91% 的机器，执行量可变减少至多 99%；TestCaseBatching 最多减少 81% 的机器，执行量减少至多 67%。作者建议实践者采用 DynamicBatching 与 TestCaseBatching 以高效降低测试机器需求，并通过历史数据分析确定增加机器收益递减的阈值。
Continuous integration at scale is costly but essential to software development. Various test optimization techniques including test selection and prioritization aim to reduce the cost. Test batching is an effective alternative, but overlooked technique. This study evaluates parallelization’s effect by adjusting machine count for test batching and introduces two novel approaches. We establish TestAll as a baseline to study the impact of parallelism and machine count on feedback time. We re-evaluate ConstantBatching and introduce DynamicBatching, which adapts batch size based on the remaining changes in the queue. We also propose TestCaseBatching, enabling new builds to join a batch before full test execution, thus speeding up continuous integration. Our evaluations utilize Ericsson’s results and 276 million test outcomes from open-source Chrome, assessing feedback time, execution reduction, and providing access to Chrome project scripts and data. The results reveal a non-linear impact of test parallelization on feedback time, as each test delay compounds across the entire test queue. ConstantBatching, with a batch size of 4, utilizes up to 72% fewer machines to maintain the actual average feedback time and provides a constant execution reduction of up to 75%. Similarly, DynamicBatching maintains the actual average feedback time with up to 91% fewer machines and exhibits variable execution reduction of up to 99%. TestCaseBatching holds the line of the actual average feedback time with up to 81% fewer machines and demonstrates variable execution reduction of up to 67%. We recommend practitioners use DynamicBatching and TestCaseBatching to reduce the required testing machines efficiently. Analyzing historical data to find the threshold where adding more machines has minimal impact on feedback time is also crucial for resource-effective testing.