一句话：首次对专有项目中 flaky 测试的复现、运行时长与修复前时间开展生命周期实证研究。
在回归测试中，开发者依赖测试通过或失败的结果判断代码改动是否破坏已有功能。flaky 测试在相同代码上非确定性地通过或失败，会给出误导性信号，因而危害显著。已有研究虽然关注 flaky 测试，但尚未探究其复现、运行时长与修复前时间等生命周期特征，也缺乏对专有项目的深入分析。本文针对这些空白，对 flaky 测试的生命周期开展实证研究，重点刻画其再次出现、执行耗时以及从发现到修复的时间间隔，并基于专有项目数据进行深入分析，从而补充工业场景下对 flaky 测试行为与维护成本的认识。
During regression testing, developers rely on the pass or fail outcomes of tests to check whether changes broke existing functionality. Thus, flaky tests, which nondeterministically pass or fail on the same code, are problematic because they provide misleading signals during regression testing. Although flaky tests are the focus of several existing studies, none of them study (1) the reoccurrence, runtimes, and time-before-fix of flaky tests, and (2) flaky tests in-depth on proprietary projects.
DOI 原文 ·
@inproceedings{paperbot5658,
title = {A study on the lifecycle of flaky tests},
author = {Wing Lam and Kıvanç Muşlu and Hitesh Sajnani and Suresh Thummalapenta},
booktitle = {ICSE},
year = {2020},
doi = {10.1145/3377811.3381749}
}