一句话：量化 Ericsson 大型项目中无故障发现（NFF）测试失败的频率，用于优先排查 flaky 测试。
大型软件系统常带着已知失败和 flaky 测试发布。本文关注一种常见但缺乏量化的现象：测试失败后，开发者经过排查却未发现故障（no-fault-found, NFF）。作者在 Ericsson 的一个大型项目上开展案例研究，覆盖四个发布版本、约 1 万项测试、990 万次测试运行，统计测试失败且未发现故障的发生频率。该工作通过工业规模的实证数据揭示 NFF 失败的普遍程度，并论证可利用这类信息对 flaky 测试的检查与排查进行优先级排序，从而更有效地分配开发者精力。
A test fails and despite an investigation by a developer there is no fault found (NFF). Large software systems are often released with known failing and flaky tests. In this work, we quantify how often a test fails and does not find a fault. We conduct a case study on 9.9 million test runs of 10k tests across four releases of a large project at Ericsson.
DOI 原文 ·
@inproceedings{paperbot6431,
title = {Quantifying no-fault-found test failures to prioritize inspection of flaky tests at Ericsson},
author = {Maaz Hafeez Ur Rehman and Peter C. Rigby},
booktitle = {FSE},
year = {2021},
doi = {10.1145/3468264.3473930}
}