一句话：提出极值反事实歧视准则，用极值理论估计最坏情况歧视，并设计采样与缓解算法。
数据驱动软件可能放大歧视，已有公平性研究多关注平均情况，忽视尾部最坏情况。本文基于极值理论提出极端反事实歧视（ECD）准则，估计仅因受保护群体身份导致的最坏结果劣势。方法上结合搜索式软件工程与生成式AI，设计随机算法，在样本不足时仍从ML结果分布尾部采样统计显著点。在四个模型、10个社会任务上评估：生成式方法在95%情况下可产生足够样本推断有效EVT分布；常见缓解器在35%情况下显著增加最坏情况歧视，MiniMax-Fairness亦在30%情况下如此。作者提出ECD缓解器，在90%情况下改善尾部公平性且不损失平均公平性。
Data-driven software is increasingly being used as a critical component of automated decision-support systems. Since this class of software learns its logic from historical data, it can encode or amplify discriminatory practices. Previous research on algorithmic fairness has focused on improving “average-case” fairness. On the other hand, fairness at the extreme ends of the spectrum, which often signifies lasting and impactful shifts in societal attitudes, has received significantly less emphasis. Leveraging the statistics of extreme value theory (EVT), we propose a novel fairness criterion called extreme counterfactual discrimination (ECD). This criterion estimates the worst-case amounts of disadvantage in outcomes for individuals solely based on their memberships in a protected group. Utilizing tools from search-based software engineering and generative AI, we present a randomized algorithm that samples a statistically significant set of points from the tail of ML outcome distributions even if the input dataset lacks a sufficient number of relevant samples. We conducted several experiments on four ML models (deep neural networks, logistic regression, and random forests) over 10 socially relevant tasks from the literature on algorithmic fairness. First, we evaluate the generative AI methods and find that they generate sufficient samples to infer valid EVT distribution in 95% of cases. Remarkably, we found that the prevalent bias mitigators reduce the average-case discrimination but increase the worst-case discrimination significantly in 35% of cases. We also observed that even the tail-aware mitigation algorithm-MiniMax-Fairness-increased the worst-case discrimination in 30% of cases. We propose a novel ECD-based mitigator that improves fairness in the tail in 90% of cases with no degradation of the average-case discrimination. We hope that the EVT framework serves as a robust tool for evaluating fairness in both average-case and worst-case discrimination.
DOI 原文 ·
@inproceedings{paperbot8688,
title = {Fairness Testing Through Extreme Value Theory},
author = {Verya Monjezi and Ashutosh Trivedi and Vladik Kreinovich and Saeid Tizpaz-Niari},
booktitle = {ICSE},
year = {2025},
doi = {10.1109/icse55347.2025.00070}
}