[Paper Review] Discrimination in the Age of Algorithms

Jon Kleinberg, J. Ludwig+2 more|arXiv (Cornell University)|Feb 11, 2019

Human Rights and Immigration4 citations

TL;DR

This paper examines how algorithms affect discrimination detection in legal and societal contexts, arguing that while algorithms can be opaque, they also enable greater transparency and auditability in decision-making processes. By enforcing specificity and traceability, well-designed algorithmic systems can make discrimination easier to detect and challenge, thus serving as tools for promoting equity when properly regulated.

ABSTRACT

The law forbids discrimination. But the ambiguity of human decision-making often makes it extraordinarily hard for the legal system to know whether anyone has actually discriminated. To understand how algorithms affect discrimination, we must therefore also understand how they affect the problem of detecting discrimination. By one measure, algorithms are fundamentally opaque, not just cognitively but even mathematically. Yet for the task of proving discrimination, processes involving algorithms can provide crucial forms of transparency that are otherwise unavailable. These benefits do not happen automatically. But with appropriate requirements in place, the use of algorithms will make it possible to more easily examine and interrogate the entire decision process, thereby making it far easier to know whether discrimination has occurred. By forcing a new level of specificity, the use of algorithms also highlights, and makes transparent, central tradeoffs among competing values. Algorithms are not only a threat to be regulated; with the right safeguards in place, they have the potential to be a positive force for equity.

Motivation & Objective

• To analyze how algorithmic decision-making influences the detection and enforcement of anti-discrimination laws.
• To investigate whether algorithms increase or reduce the risk of discrimination in practice.
• To examine the tradeoffs between transparency, fairness, and accountability in algorithmic systems.
• To identify how algorithmic systems can be designed to enhance equity and reduce bias.
• To evaluate the role of algorithms in making decision processes more scrutinizable for legal and ethical oversight.

Proposed method

• The paper uses conceptual and legal analysis to examine the interplay between algorithmic decision-making and anti-discrimination law.
• It draws on principles from computer science, artificial intelligence, and social science to assess algorithmic transparency and auditability.
• The authors examine real-world cases where algorithms were used in high-stakes decisions, such as hiring, lending, and criminal justice.
• They propose that algorithmic systems can reduce ambiguity in decision-making by making processes traceable and analyzable.
• The method emphasizes the importance of designing systems with built-in mechanisms for accountability and fairness evaluation.
• The analysis includes consideration of mathematical and cognitive opacity, and how these affect legal proof of discrimination.

Experimental results

Research questions

• RQ1 How do algorithms affect the detectability of discrimination in decision-making processes?
• RQ2 To what extent can algorithmic systems improve transparency and auditability in high-stakes decisions?
• RQ3 What are the tradeoffs between fairness, accuracy, and transparency in algorithmic systems?
• RQ4 Can algorithmic systems serve as tools for reducing bias rather than amplifying it?
• RQ5 How can legal systems adapt to use algorithmic processes as evidence in anti-discrimination cases?

Key findings

• Algorithms can reduce ambiguity in decision-making, making it easier to detect and prove discrimination compared to opaque human decisions.
• Despite their potential opacity, algorithmic systems can provide greater transparency when designed with auditability in mind.
• The use of algorithms forces decision-makers to be more specific, thereby exposing underlying tradeoffs among competing values.
• With appropriate safeguards, algorithms can become positive tools for promoting equity and reducing bias.
• Legal systems can benefit from algorithmic transparency, as it enables more rigorous scrutiny of decisions.
• The paper concludes that algorithms are not inherently discriminatory but require regulatory and design frameworks to ensure fairness.