[Paper Review] What is the Point of Fairness? Disability, AI and The Complexity of Justice

Cynthia L. Bennett, Os Keyes|arXiv (Cornell University)|Aug 2, 2019

Neuroethics, Human Enhancement, Biomedical Innovations28 references66 citations

TL;DR

The paper critiques narrow fairness framings in AI for disability, presenting two computer-vision case studies to argue for broader notions of justice that address structural injustices.

ABSTRACT

Work integrating conversations around AI and Disability is vital and valued, particularly when done through a lens of fairness. Yet at the same time, analyzing the ethical implications of AI for disabled people solely through the lens of a singular idea of "fairness" risks reinforcing existing power dynamics, either through reinforcing the position of existing medical gatekeepers, or promoting tools and techniques that benefit otherwise-privileged disabled people while harming those who are rendered outliers in multiple ways. In this paper we present two case studies from within computer vision - a subdiscipline of AI focused on training algorithms that can "see" - of technologies putatively intended to help disabled people but, through failures to consider structural injustices in their design, are likely to result in harms not addressed by a "fairness" framing of ethics. Drawing on disability studies and critical data science, we call on researchers into AI ethics and disability to move beyond simplistic notions of fairness, and towards notions of justice.

Motivation & Objective

• Motivate disability-inclusive AI research beyond single fairness concepts.
• Highlight risks of fairness alone reinforcing existing power dynamics in AI design for disabled people.
• Use disability studies and critical data science to analyze AI ethics in real-world case studies.

Proposed method

• Present two computer-vision case studies involving technologies intended to aid disabled people.
• Analyze how design decisions and lack of attention to structural injustice can cause harms despite fairness framing.
• Draw on disability studies and critical data science to critique current ethics approaches.

Experimental results

Research questions

• RQ1 How can fairness-focused AI ethics fall short in addressing structural injustices affecting disabled people?
• RQ2 What alternative frameworks of justice can better capture harms and inequalities produced by AI systems in disability contexts?
• RQ3 In what ways do current AI fairness practices and gatekeeping dynamics reproduce power imbalances?

Key findings

• Fairness alone can reinforce existing medical gatekeepers and privileged positions within disability contexts.
• AI systems intended to aid disabled people may cause harms when structural injustices are ignored in design.
• Disability studies and critical data science advocate for moving beyond fairness to more robust conceptions of justice.
• Case studies illustrate how visual AI failures fail to account for broader justice considerations.