一句话:通过归纳语义修改型变更并增强 Ekstazi 与 STARTS,使回归测试选择更精确且不损失安全性。
回归测试选择(RTS)通过仅重跑可能受代码变更影响的测试来加速回归测试。理想 RTS 需安全且精确,但常比全量重跑更慢;近期基于分析的 RTS 以精度换速度,已遇性能瓶颈。作者人工检查五个项目的代码变更,归纳出 29 类变更与 13 条发现,其中 11 条涉及语义修改。基于这些发现增强 Ekstazi 和 STARTS,使其能判定仅受此类变更影响的测试无需重跑。使用 23 个项目、1,150 个版本评估安全性与精度,并检验泛化性。结果显示,两工具平均分别少选 41.7% 和 31.8% 测试,耗时减少 33.7% 和 28.7%,且无安全性损失。
Regression test selection (RTS) speeds up regression testing by only re-running tests that might be affected by code changes. Ideal RTS safely selects all affected tests and precisely selects only affected tests. But, aiming for this ideal is often slower than re-running all tests. So, recent RTS techniques use program analysis to trade precision for speed, i.e., lower regression testing time, or even use machine learning to trade safety for speed. We seek to make recent analysis-based RTS techniques more precise, to further speed up regression testing. Independent studies suggest that these techniques reached a “performance wall” in the speed-ups that they provide. We manually inspect code changes to discover those that do not require re-running tests that are only affected by such changes. We categorize 29 kinds of changes that we find from five projects into 13 findings, 11 of which are semantics-modifying. We enhance two RTS techniques---Ekstazi and STARTS---to reason about our findings. Using 1,150 versions of 23 projects, we evaluate the impact on safety and precision of leveraging such changes. We also evaluate if our findings from a few projects can speed up regression testing in other projects. The results show that our enhancements are effective and they can generalize. On average, they result in selecting 41.7% and 31.8% fewer tests, and take 33.7% and 28.7% less time than Ekstazi and STARTS, respectively, with no loss in safety.
@inproceedings{Liu000L23,
title = {More Precise Regression Test Selection via Reasoning about Semantics-Modifying Changes},
author = {Yu Liu and Jiyang Zhang and Pengyu Nie and Milos Gligoric and Owolabi Legunsen},
booktitle = {ISSTA},
year = {2023},
doi = {10.1145/3597926.3598086}
}