一句话：提出 Parfait-ML，用搜索式测试与统计调试寻找超参数配置，在不牺牲精度的前提下改善 ML 公平性。
论文研究机器学习算法超参数选择对公平性缺陷的加剧或缓解作用。现有方法多通过修改输入数据或学习算法来处理公平性问题，而超参数提供更细粒度且侵入性较低的控制。作者设计三种基于搜索的软件测试算法，以发现超参数空间中的精度-公平性前沿，并结合统计调试解释各超参数对公平性的作用。在 Parfait-ML 工具中实现，并针对五种成熟 ML 算法、六个社会关键应用进行评估。结果表明，该方法能找到显著改善公平性且不牺牲精度的超参数；对随机森林等算法，还发现某些配置（如限制属性搜索空间）会放大偏见，并给出直观解释。
This paper investigates the parameter space of machine learning (ML) algorithms in aggravating or mitigating fairness bugs. Data-driven software is increasingly applied in social-critical applications where ensuring fairness is of paramount importance. The existing approaches focus on addressing fairness bugs by either modifying the input dataset or modifying the learning algorithms. On the other hand, the selection of hyperparameters, which provide finer controls of ML algorithms, may enable a less intrusive approach to influence the fairness. Can hyperparameters amplify or suppress discrimination present in the input dataset? How can we help programmers in detecting, understanding, and exploiting the role of hyperparameters to improve the fairness? We design three search-based software testing algorithms to un-cover the precision-fairness frontier of the hyperparameter space. We complement these algorithms with statistical debugging to explain the role of these parameters in improving fairness. We implement the proposed approaches in the tool Parfait-ML (PARameter FAIrness Testing for ML Libraries) and show its effectiveness and utility over five mature ML algorithms as used in six social-critical applications. In these applications, our approach successfully iden-tified hyperparameters that significantly improve (vis-a-vis the state-of-the-art techniques) the fairness without sacrificing precision. Surprisingly, for some algorithms (e.g., random forest), our approach showed that certain configuration of hyperparameters (e.g., restricting the search space of attributes) can amplify biases across applications. Upon further investigation, we found intuitive explanations of these phenomena, and the results corroborate simi-lar observations from the literature.
DOI 原文 · arXiv · PDF（开放获取） · DBLP
@inproceedings{Tizpaz-NiariKT022,
title = {Fairness-aware configuration of machine learning libraries},
author = {Saeid Tizpaz-Niari and Ashish Kumar and Gang Tan and Ashutosh Trivedi},
booktitle = {ICSE},
year = {2022},
doi = {10.1145/3510003.3510202}
}