[Paper Review] On Formalizing Fairness in Prediction with Machine Learning

Pratik Gajane, Pechenizkiy, Mykola|arXiv (Cornell University)|Oct 9, 2017

Ethics and Social Impacts of AISocial Sciences57 references162 citations

TL;DR

This survey analyzes how fairness is formalized in ML prediction, linking each formalization to social science notions of distributive justice, and discusses critiques and future directions.

ABSTRACT

Machine learning algorithms for prediction are increasingly being used in critical decisions affecting human lives. Various fairness formalizations, with no firm consensus yet, are employed to prevent such algorithms from systematically discriminating against people based on certain attributes protected by law. The aim of this article is to survey how fairness is formalized in the machine learning literature for the task of prediction and present these formalizations with their corresponding notions of distributive justice from the social sciences literature. We provide theoretical as well as empirical critiques of these notions from the social sciences literature and explain how these critiques limit the suitability of the corresponding fairness formalizations to certain domains. We also suggest two notions of distributive justice which address some of these critiques and discuss avenues for prospective fairness formalizations.

Motivation & Objective

• Survey how fairness is formalized in the ML prediction literature.
• Map ML fairness notions to social science distributive justice concepts.
• Critique each fairness formalization using social science literature.
• Suggest two prospective fairness notions from social sciences for future work.

Proposed method

• Review and categorize ML fairness notions (unawareness, counterfactual fairness, group fairness, individual fairness, equality of opportunity, preference-based fairness).
• Explain each notion with formal definitions and connections to distributive justice theories.
• Provide social-science critiques of each notion (biases, limitations, domain suitability).
• Discuss two prospective notions (equality of resources; equality of capability of functioning) and their potential ML formulations.
• Cross-reference related literature and empirical considerations to assess applicability across domains.

Experimental results

Research questions

• RQ1 What ML fairness formalizations exist in the literature for predictive decision-making?
• RQ2 How do these ML notions align with or conflict with distributive justice theories from social sciences?
• RQ3 What are the critiques of each fairness formalization, and in which domains might they be unsuitable?
• RQ4 What social-science notions could inspire new, more robust fairness formalizations for ML?
• RQ5 What avenues exist for integrating equality of resources or equality of capability into ML fairness frameworks?

Key findings

• Fairness formalizations differ in whether they target parity versus preferences, treatment versus impact, and group versus individual notions.
• Group fairness relies on statistical or demographic parity independent of ground truth, which can misstate merit and efficiency.
• Counterfactual fairness uses causal reasoning but may be vulnerable to biases and causality challenges.
• Individual fairness links outputs to a distance metric but depends on a non-discriminatory metric, which is hard to guarantee.
• Equality of opportunity has criticisms about not addressing broader social influences on life prospects.
• The authors propose equality of resources and equality of capability of functioning as prospective fairness notions to address limitations.