一句话:通过实证研究发现扩大特征集可同时提升公平性与准确率,而增大训练数据未必改善公平性。
机器学习软件在涉及人的决策中可能对不同群体产生偏见。现有研究主要关注公平性指标与改进方法,但对特征集和训练数据等关键因素如何影响公平性仍不清楚。本文开展综合性实证研究,系统考察特征集规模与训练数据规模对软件公平性和准确率的影响。结果表明,扩大特征集对公平性有显著作用,平均效应率为38%,且与广泛认为的公平性提升会牺牲准确率相反,扩大特征集还能同时提高准确率。令人意外的是,更大的训练数据并不改善公平性;当特征集不足时,较大训练数据甚至比较小训练数据更不公平。该发现对软件工程实践具有警示意义。
Machine learning software can be unfair when making human-related decisions, having prejudices over certain groups of people. Existing work primarily focuses on proposing fairness metrics and presenting fairness improvement approaches. It remains unclear how key aspect of any machine learning system, such as feature set and training data, affect fairness. This paper presents results from a comprehensive study that addresses this problem. We find that enlarging the feature set plays a significant role in fairness (with an average effect rate of 38%). Importantly, and contrary to widely-held beliefs that greater fairness often corresponds to lower accuracy, our findings reveal that an enlarged feature set has both higher accuracy and fairness. Perhaps also surprisingly, we find that a larger training data does not help to improve fairness. Our results suggest a larger training data set has more unfairness than a smaller one when feature sets are insufficient; an important cautionary finding for practising software engineers.