一句话：提出 Croissant 框架，用 18 个 flakiness 变异算子向测试套件注入不确定性，生成高质量 flaky test 基准以评估检测工具。
回归测试是保障代码变更不引入缺陷的常用手段，但 flaky test（无论代码是否变更都非确定性地通过或失败）会削弱其有效性。当前社区在 flakiness 检测与缓解技术上不断进步，却缺少系统化生成高质量 flaky test 基准的方法来比较这些技术。受变异测试评估测试故障检测能力的启发，本文提出 Croissant：一个向测试套件注入 flakiness 的框架，用于评估 flakiness 检测工具发现此类测试的效果。Croissant 实现了 18 个可诱发 flakiness 的变异算子，并设计这些算子以控制其中涉及的非确定性，即让大量变异体确定性地通过或失败，从而可观察 flaky 行为。作者在 15 个真实项目的测试套件上进行了广泛的实证评估，结果证实所设计的变异算子能够生成高质量的变异体，并能有效挑战 flakiness 检测工具、暴露其中的 flaky test。
Software developers often rely on regression testing to ensure that recent changes made to the source code do not introduce bugs. Flaky tests, which non-deterministically pass or fail regardless of any change to the code, can negatively impact the effectiveness of the regression testing. While state-of-the-art is advancing the techniques for test-flakiness detection and mitigation, the community is missing a systematic approach for generating high-quality benchmarks of flaky tests to compare the effectiveness of such techniques. Inspired by the power of mutation testing in evaluating the fault-detection ability of tests, this paper proposes Croissant, a framework for injecting flakiness into the test suites to assess the effectiveness of test-flakiness detection tools in finding these tests. Croissant implements 18 flakiness-inducing mutation operators. We designed these operators to allow controlling the non-determinism involved in flakiness, i.e., making many mutants deterministically pass or fail to observe flaky behavior. Our extensive empirical evaluation of Croissant on the test suites of 15 real-world projects confirms the ability of designed mutation operators to generate high-quality mutants, and their effectiveness in challenging test-flakiness detection tools in revealing flaky tests.