一句话：提出 FLEX，通过更新断言边界修复机器学习项目中因随机性导致的 flaky 测试。
许多机器学习算法具有内在随机性，即使输入相同，多次执行也可能产生略有差异的结果。这种随机性给测试带来困难：开发者在检查端到端质量时，需要将获得的指标与参考结果比较，而选择合适阈值并非直观任务，阈值不当会导致 flaky test execution。标题表明论文提出 FLEX，其核心思路是更新断言边界，以修复机器学习项目中的 flaky 测试。给定摘要未提供方法的详细算法、实验设置与修复效果，因此只能概述其问题定位与高层修复方向。
Many machine learning (ML) algorithms are inherently random – multiple executions using the same inputs may produce slightly different results each time. Randomness impacts how developers write tests that check for end-to-end quality of their implementations of these ML algorithms. In particular, selecting the proper thresholds for comparing obtained quality metrics with the reference results is a non-intuitive task, which may lead to flaky test executions.
DOI 原文 ·
@inproceedings{paperbot6393,
title = {FLEX: fixing flaky tests in machine learning projects by updating assertion bounds},
author = {Saikat Dutta and August Shi and Sasa Misailovic},
booktitle = {FSE},
year = {2021},
doi = {10.1145/3468264.3468615}
}