一句话：将传统软件测试规划与组合建模用于 ML 方案，借助特征交互与数据切片覆盖业务需求并强化薄弱区域。
本文关注 ML 解决方案测试中过度优化 F1 分数、准确率等 ML 指标而忽视业务需求覆盖的问题。作者将经典软件的测试规划方法迁移到 ML 解决方案：使用组合建模方法定义业务需求空间，并将其映射到 ML 解决方案的数据；再利用数据切片识别 ML 方案的薄弱区域并加以强化。作者将该方法应用于三个真实世界案例研究，展示了其在弥合 ML 方案与业务需求差距方面的价值。
Machine Learning (ML) based solutions are becoming increasingly popular and pervasive. When testing such solutions, there is a tendency to focus on improving the ML metrics such as the F1-score and accuracy at the expense of ensuring business value and correctness by covering business requirements. In this work, we adapt test planning methods of classical software to ML solutions. We use combinatorial modeling methodology to define the space of business requirements and map it to the ML solution data, and use the notion of data slices to identify the weaker areas of the ML solution and strengthen them. We apply our approach to three real-world case studies and demonstrate its value.
DOI 原文 ·
@inproceedings{paperbot5077,
title = {Bridging the gap between ML solutions and their business requirements using feature interactions},
author = {Guy Barash and Eitan Farchi and Ilan Jayaraman and Orna Raz and Rachel Tzoref-Brill and Marcel Zalmanovici},
booktitle = {FSE},
year = {2019},
doi = {10.1145/3338906.3340442}
}