一句话：聚焦大型工业 CI 中 flaky tests 的根因定位，分析非确定性失败对开发者生产率与可靠性的影响。
论文研究大型工业环境下 flaky tests 的根因定位问题。CI 流水线中，同一代码版本的测试可能时通过时失败；这种非确定性失败不仅浪费开发者大量排查时间，还可能被误认为与本次变更无关而忽略。然而，忽略 flaky 失败也有风险，因为其中可能隐藏生产代码的真实缺陷。作者指出，flakiness 常由并发、外部依赖等意外且非确定性的行为导致，使其根因诊断繁琐困难。本文以此为背景，面向大规模工业场景探讨如何定位 flaky tests 的根本原因，以提升 CI 可靠性和开发者生产率。
In today’s agile world, developers often rely on continuous integration pipelines to help build and validate their changes by executing tests in an efficient manner. One of the significant factors that hinder developers’ productivity is flaky tests—tests that may pass and fail with the same version of code. Since flaky test failures are not deterministically reproducible, developers often have to spend hours only to discover that the occasional failures have nothing to do with their changes. However, ignoring failures of flaky tests can be dangerous, since those failures may represent real faults in the production code. Furthermore, identifying the root cause of flakiness is tedious and cumbersome, since they are often a consequence of unexpected and non-deterministic behavior due to various factors, such as concurrency and external dependencies.
DOI 原文 ·
@inproceedings{paperbot5391,
title = {Root causing flaky tests in a large-scale industrial setting},
author = {Wing Lam and Patrice Godefroid and Suman Nath and Anirudh Santhiar and Suresh Thummalapenta},
booktitle = {ISSTA},
year = {2019},
doi = {10.1145/3293882.3330570}
}