一句话：提出统一方法学，仅用 CI 与版本控制元数据构建并评估回归测试选择与优先级排序方法，并在 23 个项目上开展大规模实证。
回归测试选择（RTS）与优先级排序（RTP）可减少测试开销与反馈时间，但其常用信息源（测试轨迹、构建依赖、测试历史等）并非在所有持续集成环境中都可获得，而版本控制系统与 CI 系统的元数据则廉价且普遍可得。本文把此前分散的 RTP 与不安全 RTS 研究整合为一套可操作的方法学，用于构建和评估仅依赖 CI 与 VCS 元数据的方法，并在 23 个项目、37,000 份 CI 日志和 76,000 次提交上进行大规模实证。结果显示，这类方法显著优于既有 RTP 基线；不安全 RTS 虽然仍会触发 90% 的失败，但平均可节省 84% 的测试执行时间。研究还发现限制训练数据可能有益，测试历史特征优于基于变更的特征，且简单经典的启发式规则往往胜过复杂的机器学习模型。
Regression test selection (RTS) and prioritization (RTP) techniques aim to reduce testing efforts and developer feedback time after a change to the code base. Using various information sources, including test traces, build dependencies, version control data, and test histories, they have been shown to be effective. However, not all of these sources are guaranteed to be available and accessible for arbitrary continuous integration (CI) environments. In contrast, metadata from version control systems (VCSs) and CI systems are readily available and inexpensive. Yet, corresponding RTP and RTS techniques are scattered across research and often only evaluated on synthetic faults or in a specific industrial context. It is cumbersome for practitioners to identify insights that apply to their context, let alone to calibrate associated parameters for maximum cost-effectiveness. This paper consolidates existing work on RTP and unsafe RTS into an actionable methodology to build and evaluate such approaches that exclusively rely on CI and VCS metadata. To investigate how these approaches from prior research compare in heterogeneous settings, we apply the methodology in a large-scale empirical study on a set of 23 projects covering 37,000 CI logs and 76,000 VCS commits. We find that these approaches significantly outperform established RTP baselines and, while still triggering 90% of the failures, we show that practitioners can expect to save on average 84% of test execution time for unsafe RTS. We also find that it can be beneficial to limit training data, features from test history work better than change-based features, and, somewhat surprisingly, simple and well-known heuristics often outperform complex machine-learned models.
@inproceedings{paperbot6651,
title = {Empirically evaluating readily available information for regression test optimization in continuous integration},
author = {Daniel Elsner and Florian Hauer and Alexander Pretschner and Silke Reimer},
booktitle = {ISSTA},
year = {2021},
doi = {10.1145/3460319.3464834}
}