一句话：提出基于因果的测试框架，评估公平性最佳实践在标签错误、数据缺失与分布偏移下的鲁棒性。
机器学习算法广泛部署于金融、刑事司法等社会经济关键决策，但可能因数据驱动特性产生不公平决策逻辑，损害边缘群体。公平性最佳实践（如包含敏感特征、选择非敏感属性、应用偏差缓解器）在标签错误、数据缺失或分布偏移下是否可靠尚不明确。本文提出基于因果理论的测试工具与技术，将实践表示为一阶逻辑性质，对满足该性质的社会关键数据集，通过因果图表示与搜索算法探索等价因果图，生成改变因果假设的邻域数据集，检验性质是否继续成立。评估覆盖多个公平敏感任务，识别出在不同干扰因素下仍保持鲁棒的最佳实践。
Machine learning (ML) algorithms are increasingly deployed to make critical decisions in socioeconomic applications such as finance, criminal justice, and autonomous driving. However, due to their data-driven and pattern-seeking nature, ML algorithms may develop decision logic that disproportionately distributes opportunities, benefits, resources, or information among different population groups, potentially harming marginalized communities. In response to such fairness concerns, the software engineering and ML communities have made significant efforts to establish the best practices for creating fair ML software. These include fairness interventions for training ML models, such as including sensitive features, selecting non-sensitive attributes, and applying bias mitigators. But how reliably can software professionals tasked with developing data-driven systems depend on these recommendations? And how well do these practices generalize in the presence of faulty labels, missing data, or distribution shifts? These questions form the core theme of this paper. We present a testing tool and technique based on causality theory to assess the robustness of best practices in fair ML software development. Given a practice—specified as a first-order logic property— and a socio-critical dataset that satisfies the property, our goal is to search for neighborhood datasets to determine whether the property continues to hold. This process is akin to testing the robustness of a neural network for image classification, except that the “image" is an entire dataset, and its “neighbors" are datasets in which certain causal hypotheses are altered. Since computing neighborhood datasets while accounting for various factors—such as noise, faulty labeling, and demographic shifts—is challenging, we utilize causal graph representations of the dataset and leverage a search algorithm to explore equivalent causal graphs to generate datasets. Our results across various fairness-sensitive tasks, derived from prevalent fairness-sensitive applications, identify best practices that preserve robustness under the varying factors.
@inproceedings{abs-2601-03621,
title = {On the Robustness of Fairness Practices: A Causal Framework for Systematic Evaluation},
author = {Verya Monjezi and Ashish Kumar and Ashutosh Trivedi and Gang Tan and Saeid Tizpaz-Niari},
booktitle = {ICSE},
year = {2026},
doi = {10.1145/3744916.3773175}
}