一句话：提出以最小可检测效应（MDE）为核心的可检测性引导微基准执行框架，在统计保证下大幅缩短性能回归测试时间。
性能微基准广泛用于持续集成中的性能回归检测，但现有执行策略计算开销高昂。本文将基准执行重构为最小可检测效应（MDE）问题，引入双阈值规范：下阈值 δ0 对应可忽略的减速，上阈值 δ1 对应有意义的回归，并从理论与实证两方面证明该规范可将假阳性率控制在 δ0 以下、在 δ1 附近保证统计功效，同时显著缩短执行时间。为实现该框架，作者提出两阶段流程：第一阶段基于 MDE 在线重估样本量自适应控制执行，在达到足够统计分辨率时提前停止，在预算内无法达到目标可检测性时及时终止；第二阶段在同一规范下进行回归分析，给出具有可证明误差界的判定。在来自三个大型开源项目的 1702 个微基准上，执行时间最多减少 93%，经验假阳性率与统计功效与理论目标高度吻合。
Performance microbenchmarks are widely used for detecting performance regressions in continuous integration, yet existing execution policies remain computationally expensive. We present detectability-guided microbenchmarking, a principled framework that reformulates benchmark execution as a Minimum-Detectable-Effect (MDE) problem, achieving substantial speedups with formal statistical guarantees. Our approach introduces a two-threshold specification: a lower threshold, δ0, for negligible slowdowns, and an upper threshold, δ1, for meaningful regressions. We theoretically and empirically show that this specification enables bounded false-positive rates below δ0 and guaranteed statistical power around δ1, while dramatically reducing execution time. To realize this framework, we propose a two-phase procedure: Phase 1 adaptively controls execution via online sample-size re-estimation guided by minimum detectable effect, enabling early stop when sufficient statistical resolution is reached and futile abort when target detectability is unattainable within the execution budget; Phase 2 performs regression analysis under the same specification, yielding decisions with provable error bounds. Experiments on 1702 microbenchmarks from three large-scale open-source projects demonstrate up to 93% reduction in execution time while empirical false-positive rates and power closely match their theoretical targets. Our findings reveal that detectability-guided execution offers a theoretically grounded yet practical path toward efficient microbenchmark-based regression detection.
@inproceedings{paperbot12353,
title = {Stop When It Matters: Detectability-Guided Microbenchmarking for Performance Regression Testing},
author = {Zongxiong Chen and Derui Zhu and Jinfu Chen and Kundi Yao and Alexander Pretschner and Weiyi Shang and Manfred Hauswirth and Sonja Schimmler},
booktitle = {ASE},
year = {2026},
doi = {10.1145/3832783.3834367}
}