一句话:指出现有工作缺乏对已训练机器学习模型、尤其是黑盒模型的单调性检查方法,并聚焦其测试需求。
机器学习模型日益用于贷款等决策场景,因此需要针对领域相关需求进行质量保证。单调性是一种常见需求:当某些属性值增大时,模型预测也应单调增大(例如收入越高,贷款额度不应更低)。已有多种 ML 算法可在训练阶段生成满足单调性的模型,但如何对已生成的模型、尤其是无法访问内部结构的黑盒模型检查单调性,仍缺乏系统方法。该论文聚焦这一缺口,围绕机器学习模型的单调性测试展开,强调黑盒场景下检查单调性的必要性。摘要未给出具体算法或实验结果的细节,因此其核心贡献在于明确问题并推动针对黑盒 ML 模型的单调性测试。
Today, machine learning (ML) models are increasingly applied in decision making. This induces an urgent need for quality assurance of ML models with respect to (often domain-dependent) requirements. Monotonicity is one such requirement. It specifies a software as ''learned'' by an ML algorithm to give an increasing prediction with the increase of some attribute values. While there exist multiple ML algorithms for ensuring monotonicity of the generated model, approaches for checking monotonicity, in particular of black-box models are largely lacking.
DOI 原文 ·
@inproceedings{paperbot6069,
title = {Higher income, larger loan? monotonicity testing of machine learning models},
author = {Arnab Sharma and Heike Wehrheim},
booktitle = {ISSTA},
year = {2020},
doi = {10.1145/3395363.3397352}
}