[Paper Review] Are There Exceptions to Goodhart's Law? On the Moral Justification of Fairness-Aware Machine Learning

Hilde Weerts, Lambèr Royakkers+1 more|arXiv (Cornell University)|Feb 17, 2022

Ethics and Social Impacts of AI4 citations

TL;DR

This paper develops a moral reasoning framework to evaluate when fairness-aware machine learning (fair-ml) techniques are justifiable, arguing that fairness metrics like demographic parity and equalized odds only justify interventions under specific conditions involving individual utility and moral claims to benefits or burdens. It shows that optimizing for these metrics via algorithms—especially post-processing methods like Hardt et al. (2016)—can lead to unjust outcomes due to side effects like leveling down and inconsistency, even when metrics are satisfied.

ABSTRACT

Fairness-aware machine learning (fair-ml) techniques are algorithmic interventions designed to ensure that individuals who are affected by the predictions of a machine learning model are treated fairly. The problem is often posed as an optimization problem, where the objective is to achieve high predictive performance under a quantitative fairness constraint. However, any attempt to design a fair-ml algorithm must assume a world where Goodhart's law has an exception: when a fairness measure becomes an optimization constraint, it does not cease to be a good measure. In this paper, we argue that fairness measures are particularly sensitive to Goodhart's law. Our main contributions are as follows. First, we present a framework for moral reasoning about the justification of fairness metrics. In contrast to existing work, our framework incorporates the belief that whether a distribution of outcomes is fair, depends not only on the cause of inequalities but also on what moral claims decision subjects have to receive a particular benefit or avoid a burden. We use the framework to distil moral and empirical assumptions under which particular fairness metrics correspond to a fair distribution of outcomes. Second, we explore the extent to which employing fairness metrics as a constraint in a fair-ml algorithm is morally justifiable, exemplified by the fair-ml algorithm introduced by Hardt et al. (2016). We illustrate that enforcing a fairness metric through a fair-ml algorithm often does not result in the fair distribution of outcomes that motivated its use and can even harm the individuals the intervention was intended to protect.

Motivation & Objective

• To address the lack of moral guidance in selecting and applying fairness metrics in fair-ml techniques.
• To analyze whether fairness metrics like demographic parity and equalized odds can be morally justified under specific conditions.
• To investigate the moral implications of post-processing algorithms, particularly Hardt et al. (2016), which optimize fairness through group-specific or randomized decision thresholds.
• To challenge the assumption that satisfying a fairness metric ensures a fair distribution of benefits and burdens.
• To advocate for a holistic evaluation of fair-ml algorithms beyond their optimization objectives, incorporating procedural and distributive justice.

Proposed method

• The authors extend a framework from Hertweck et al. (2021) by integrating the utility of predicted outcomes for decision subjects and their moral claims to benefits or burdens.
• They apply this extended framework to evaluate two fairness metrics—demographic parity and equalized odds—across three stylized real-world examples involving loan approval, hiring, and healthcare.
• The paper analyzes the Hardt et al. (2016) post-processing algorithm, which adjusts decision thresholds per group to equalize misclassification rates.
• It distinguishes between two optimization strategies: group-specific deterministic thresholds and randomized thresholds to balance group-level fairness.
• The analysis evaluates moral objections such as 'leveling down' and violations of consistency and proportionality in individual treatment.
• The authors use counterexamples to demonstrate how satisfying fairness metrics can still lead to unjust outcomes due to unintended side effects.

Experimental results

Research questions

• RQ1 Under what conditions is the use of a fairness metric like demographic parity morally justifiable?
• RQ2 How do individual utility and moral claims to benefits or burdens affect the moral justification of fairness metrics?
• RQ3 In what circumstances does optimizing for a fairness metric via a fair-ml algorithm lead to unjust outcomes despite metric satisfaction?
• RQ4 What are the moral implications of using group-specific or randomized decision thresholds in post-processing fair-ml algorithms?
• RQ5 When should fair-ml techniques be avoided altogether, even if they satisfy formal fairness criteria?

Key findings

• Demographic parity and equalized odds are only morally justifiable when the utility of outcomes for decision subjects and their moral claims to benefits or burdens are properly aligned with the fairness metric.
• Optimizing for fairness metrics through post-processing algorithms like Hardt et al. (2016) can result in the 'leveling down' objection, where individuals in more privileged groups are treated worse than they would be without intervention.
• Randomized group-specific decision thresholds risk violating the principle of consistency, as they may assign random predictions regardless of individual characteristics, undermining fairness at the individual level.
• Even when fairness metrics are satisfied, the resulting distribution of outcomes may not reflect a fair allocation of benefits and burdens due to oversimplified operationalizations of fairness.
• The use of fairness metrics as optimization objectives can introduce harmful side effects, such as reduced predictive accuracy and inconsistent treatment, which undermine the very goals of fairness-aware machine learning.
• The study concludes that technical choices in fair-ml are not neutral, but carry deep moral implications, necessitating transparency and procedural justice in algorithmic design.