[论文解读] Fairness-Accuracy Trade-Offs: A Causal Perspective

Drago Plečko, Elias Bareinboim|arXiv (Cornell University)|May 24, 2024

Climate Change Policy and EconomicsEconomics, Econometrics and Finance被引用 3

一句话总结

本文提出一种因果框架，通过引入路径特异性超额损失（PSEL）来量化在单条因果路径上强制实现公平性所带来的准确率损失，从而分析机器学习中的公平性-准确率权衡。该方法将总超额损失分解为局部PSEL，提出因果公平性/效用比以比较不同路径上的权衡，并开发了一种基于神经网络的因果约束公平学习方法，实证表明在真实世界数据集中，公平性与准确率始终存在矛盾。

ABSTRACT

Systems based on machine learning may exhibit discriminatory behavior based on sensitive characteristics such as gender, sex, religion, or race. In light of this, various notions of fairness and methods to quantify discrimination were proposed, leading to the development of numerous approaches for constructing fair predictors. At the same time, imposing fairness constraints may decrease the utility of the decision-maker, highlighting a tension between fairness and utility. This tension is also recognized in legal frameworks, for instance in the disparate impact doctrine of Title VII of the Civil Rights Act of 1964 -- in which specific attention is given to considerations of business necessity -- possibly allowing the usage of proxy variables associated with the sensitive attribute in case a high-enough utility cannot be achieved without them. In this work, we analyze the tension between fairness and accuracy from a causal lens for the first time. We introduce the notion of a path-specific excess loss (PSEL) that captures how much the predictor's loss increases when a causal fairness constraint is enforced. We then show that the total excess loss (TEL), defined as the difference between the loss of predictor fair along all causal pathways vs. an unconstrained predictor, can be decomposed into a sum of more local PSELs. At the same time, enforcing a causal constraint often reduces the disparity between demographic groups. Thus, we introduce a quantity that summarizes the fairness-utility trade-off, called the causal fairness/utility ratio, defined as the ratio of the reduction in discrimination vs. the excess loss from constraining a causal pathway. This quantity is suitable for comparing the fairness-utility trade-off across causal pathways. Finally, as our approach requires causally-constrained fair predictors, we introduce a new neural approach for causally-constrained fair learning.

研究动机与目标

• 从因果视角分析机器学习中公平性与准确率之间的权衡，解决传统统计公平性定义的局限性。
• 通过一种名为路径特异性超额损失（PSEL）的新度量，量化在特定因果路径上强制实现公平性所带来的准确率成本。
• 将公平性约束导致的总超额损失分解为各条因果路径的贡献，实现对公平性-效用权衡的细粒度分析。
• 引入因果公平性/效用比作为比较指标，用于评估不同因果路径上公平性干预措施的效率。
• 开发一种神经网络架构，用于学习因果约束下的公平预测器，实现所提框架在实际中的部署。

提出的方法

• 提出路径特异性超额损失（PSEL）作为衡量指标，用于衡量当特定因果路径被约束为公平时，预测器损失的增加程度。
• 推导出总超额损失（TEL）在所有因果路径上的PSEL之和的分解形式，实现对准确率成本在各条路径上的归因。
• 引入因果公平性/效用比，即歧视减少量与超额损失的比值，用于比较不同路径上公平性干预措施的优劣。
• 开发一种基于神经网络的方法，通过修改反向传播算法以在指定因果路径上强制实现公平性，从而训练因果约束下的公平预测器。
• 应用一种算法（算法1），系统评估移除每条因果路径对公平性（以TVD衡量）和准确率（以损失衡量）的影响，结合PSEL和CFUR指标进行分析。
• 使用因果图可视化每条路径对公平性和准确性的贡献，并在图上标注PSEL和TVD数值。

Figure 1 : TV vs. Excess Loss.

实验结果

研究问题

• RQ1 如何在单条因果路径上而非全局范围内量化公平性-准确率权衡？
• RQ2 在特定因果路径上强制实现公平性会使预测器损失增加多少，这种成本如何衡量？
• RQ3 哪些因果路径提供了最有利的公平性-效用权衡，如何对它们进行比较？
• RQ4 是否可以训练一个神经网络，使其在保持预测性能的同时，对特定因果路径强制实现公平性？
• RQ5 在现实世界的机器学习应用中，不同因果路径如何对群体差异产生影响？

主要发现

• 在COMPAS再犯预测数据集中，移除虚假效应使群体差异减少了3%（TVD），而移除间接效应使差异减少了5%，表明移除间接效应能带来更强的公平性收益。
• COMPAS数据集中，间接效应的路径特异性超额损失（PSEL）占AUROC的12%，显著高于虚假效应的6%，表明对间接效应施加约束带来的准确率成本更大。
• 因果公平性/效用比（CFUR）分析表明，在德国信贷数据集中，移除直接效应在公平性增益与准确率损失之间实现了最佳权衡。
• 总超额损失（TEL）成功分解为各条路径PSEL之和，验证了理论分解（定理1）。
• 所提出的神经方法用于因果约束公平学习，成功构建了可使用PSEL和CFUR框架在多个数据集上评估的公平预测器。
• 在三个真实世界数据集（德国信贷、COMPAS、随机森林）上的实证结果一致表明，公平性与准确率存在矛盾，没有任何一条路径能在不付出准确率代价的情况下实现完美公平。

(a) (A)PSEL and (A)TVD values.