一句话：提出可扩展的 pytest 插件 FlakeFighters，利用静态与执行数据将 Python 测试失败分类为真正缺陷或不稳定行为。
不稳定测试（flaky tests）对同一代码版本会非确定性地通过或失败，给开发者带来不可预测的困扰。当前实践通常直接重跑测试来检测和规避 flaky 测试，但这会侵蚀失败结果的可信度，并浪费大量时间与计算资源。更好的方案是结合静态数据（无需运行测试即可收集）和/或执行数据（运行测试套件收集）来判断测试失败是源于 flaky 行为还是真实缺陷。已有若干此类技术，但它们多为孤立的研究原型工具，且通常仅支持 Java 代码库。本文提出 FlakeFighters，一个可扩展的 pytest 扩展，实现了 flaky 测试分类工具的“瑞士军刀”，帮助开发者将 Python 代码库中失败测试的根因分类为真实缺陷或不稳定性。作者在 home-assistant/core 仓库的 25 个失败工作流上运行带 FlakeFighters 的 pytest，发现它能正确识别所有失败的不稳定测试。
This is the replication package for our evaluation for our tool paper entitled "Flakefighters: A PyTest Plugin to Detect Flaky Test Failures". The extension can be found at https://pypi.org/project/pytest-flakefighters. To use this data, first download the experimental code. Then unpack the data zip file into the "outputs" directory.Flaky tests — tests which non-deterministically pass and fail against the same version of code — pose a well-established painful issue for developers due to the unpredictability of failures. The current state of practice to detect and work around flaky tests is to simply rerun them. However, this can erode confidence in failing test outcomes and represents a significant waste of time and compute resources. A better solution is to detect flaky tests based on static (data collectable without running any tests) and/or execution data (data collected by running the test suite) to determine if a test failure has occurred due to flaky behaviour or a genuine bug. Several such techniques have appeared in recent literature, but these exist in isolation as prototype research tools, and typically only support Java codebases. This paper presents FlakeFighters, an extensible pytest extension that implements a “Swiss army knife” of flaky test classification tools to help developers classify the underlying cause of failing tests as genuine bugs or flakiness in Python codebases. We ran pytest with the FlakeFighters extension on 25 failing workflows from the home-assistant/core repository and found that it was able to correctly identify all of the failing flaky tests. A video demonstration of FlakeFighters can be found at https://youtu.be/3J4odtgzQX0.
@inproceedings{paperbot12430,
title = {FlakeFighters: A Pytest Plugin to Detect Flaky Test Failures},
author = {Michael Foster and Owain Parry and Michael Hilton and Gregory M. Kapfhammer and Phil McMinn},
booktitle = {ASE},
year = {2026},
doi = {10.1145/3832783.3834641}
}