一句话：提出 LIMI，借助 GAN 语义隐空间模仿目标模型决策边界，生成更自然高效的个体歧视性实例用于黑盒公平性测试。
机器学习系统在招聘、信贷等敏感领域常表现出不公平行为，现有公平性测试方法生成的个体歧视性实例偏离真实数据分布、缺乏自然性，难以反映模型实际公平性。本文提出 Latent Imitator（LIMI）框架：在 GAN 的语义隐空间中模仿目标模型的决策边界，先推导一个代理线性边界来粗略近似目标模型决策边界并保留原始数据分布特性，再将随机隐向量一步移动到该代理边界上，并通过向量计算探测两个可能更接近真实决策边界的歧视性候选实例。在多个数据集上的实验表明，LIMI 在有效性（×9.42 实例）、效率（×8.71 加速）和自然性（+19.65%）上平均大幅优于基线；用其生成的测试样本重训练还可提升个体公平性（IFr 45.67%、IFo 32.81%）与群组公平性（SPD 9.86%、AOD 28.38%）。
Machine learning (ML) systems have achieved remarkable performance across a wide area of applications. However, they frequently exhibit unfair behaviors in sensitive application domains (e.g., employment and loan), raising severe fairness concerns. To evaluate and test fairness, engineers often generate individual discriminatory instances to expose unfair behaviors before model deployment. However, existing baselines ignore the naturalness of generation and produce instances that deviate from the real data distribution, which may fail to reveal the actual model fairness since these unnatural discriminatory instances are unlikely to appear in practice. To address the problem, this paper proposes a framework named Latent Imitator (LIMI) to generate more natural individual discriminatory instances with the help of a generative adversarial network (GAN), where we imitate the decision boundary of the target model in the semantic latent space of GAN and further samples latent instances on it. Specifically, we first derive a surrogate linear boundary to coarsely approximate the decision boundary of the target model, which reflects the nature of the original data distribution. Subsequently, to obtain more natural instances, we manipulate random latent vectors to the surrogate boundary with a one-step movement, and further conduct vector calculation to probe two potential discriminatory candidates that may be more closely located in the real decision boundary. Extensive experiments on various datasets demonstrate that our LIMI outperforms other baselines largely in effectiveness (×9.42 instances), efficiency (×8.71 speeds), and naturalness (+19.65%) on average. In addition, we empirically demonstrate that retraining on test samples generated by our approach can lead to improvements in both individual fairness (45.67% on IFr and 32.81% on IFo) and group fairness (9.86% on SPD and 28.38% on AOD). Our codes can be found on our website.
DOI 原文 · arXiv · PDF（开放获取） · DBLP
@inproceedings{XiaoLL023,
title = {Latent Imitator: Generating Natural Individual Discriminatory Instances for Black-Box Fairness Testing},
author = {Yisong Xiao and Aishan Liu and Tianlin Li and Xianglong Liu},
booktitle = {ISSTA},
year = {2023},
doi = {10.1145/3597926.3598099}
}