一句话：提出黑盒个体公平性测试方法MAFT，用零阶梯度估计与属性扰动高效发现DNN公平性违背。
深度神经网络在决策系统中广泛应用，但其公平性问题持续受到关注。现有白盒个体公平性测试方法较为有效，而黑盒方法发展停滞、性能明显落后。本文提出模型无关公平性测试方法MAFT，面向黑盒场景识别深度模型中的个体公平性违背。该方法采用零阶梯度估计与属性扰动等轻量过程，避免符号执行等高开销技术，从而提升可扩展性和适用性。实验表明，MAFT在有效性上可与当前最优白盒方法相当，同时更适用于大规模网络。与现有黑盒方法相比，其在发现公平性违背方面有效性和效率分别约提升14.69倍和32.58倍。
Deep neural networks (DNNs) have shown powerful performance in various applications and are increasingly being used in decisionmaking systems. However, concerns about fairness in DNNs always persist. Some efficient white-box fairness testing methods about individual fairness have been proposed. Nevertheless, the development of black-box methods has stagnated, and the performance of existing methods is far behind that of white-box methods. In this paper, we propose a novel black-box individual fairness testing method called Model-Agnostic Fairness Testing (MAFT). By leveraging MAFT, practitioners can effectively identify and address discrimination in DL models, regardless of the specific algorithm or architecture employed. Our approach adopts lightweight procedures such as gradient estimation and attribute perturbation rather than non-trivial procedures like symbol execution, rendering it significantly more scalable and applicable than existing methods. We demonstrate that MAFT achieves the same effectiveness as state-of-the-art white-box methods whilst improving the applicability to large-scale networks. Compared to existing black-box approaches, our approach demonstrates distinguished performance in discovering fairness violations w.r.t effectiveness (~ 14.69×) and efficiency (~ 32.58×).
@inproceedings{paperbot11477,
title = {MAFT: Efficient Model-Agnostic Fairness Testing for Deep Neural Networks via Zero-Order Gradient Search},
author = {Zhaohui Wang and Min Zhang and Jingran Yang and Bojie Shao and Min Zhang},
booktitle = {ICSE},
year = {2024},
doi = {10.1145/3597503.3639181}
}