[Paper Review] The Theory of Artificial Immutability: Protecting Algorithmic Groups Under Anti-Discrimination Law

Sandra Wachter|arXiv (Cornell University)|May 2, 2022

Discrimination and Equality Law11 citations

TL;DR

This paper introduces the 'theory of artificial immutability' to extend anti-discrimination law protections to algorithmic groups—social categories derived from AI systems that are not legally protected but function like immutable characteristics. By demonstrating how these groups constrain autonomy and life outcomes, the theory argues for legal recognition under existing non-discrimination frameworks, especially in North American and European jurisdictions.

ABSTRACT

Artificial Intelligence (AI) is increasingly used to make important decisions about people. While issues of AI bias and proxy discrimination are well explored, less focus has been paid to the harms created by profiling based on groups that do not map to or correlate with legally protected groups such as sex or ethnicity. This raises a question: are existing equality laws able to protect against emergent AI-driven inequality? This article examines the legal status of algorithmic groups in North American and European non-discrimination doctrine, law, and jurisprudence and will show that algorithmic groups are not comparable to traditional protected groups. Nonetheless, these new groups are worthy of protection. I propose a new theory of harm - "the theory of artificial immutability" - that aims to bring AI groups within the scope of the law. My theory describes how algorithmic groups act as de facto immutable characteristics in practice that limit people's autonomy and prevent them from achieving important goals.

Motivation & Objective

• To examine whether existing anti-discrimination laws in North America and Europe can protect individuals from harms caused by algorithmic groups.
• To identify the legal and practical limitations of current non-discrimination doctrine in addressing AI-driven profiling based on non-traditional, non-protected groups.
• To propose a new legal theory—'artificial immutability'—that frames algorithmic groups as de facto immutable characteristics despite lacking formal legal status.
• To demonstrate how algorithmic groups restrict personal autonomy and prevent individuals from achieving key life goals, mirroring harms associated with legally protected characteristics.
• To advocate for the integration of algorithmic groups into anti-discrimination law through a novel legal framework grounded in real-world impacts rather than formal legal categories.

Proposed method

• Analyzes case law, legal doctrine, and jurisprudence from North American and European legal systems to assess the treatment of non-traditional social groups.
• Identifies the structural and functional similarities between algorithmic groups and legally protected, immutable characteristics such as sex or race.
• Constructs a legal theory based on the practical consequences of algorithmic group membership—particularly the inability to change or escape group identity in practice.
• Applies the concept of 'immutability' not as a legal category but as a functional reality: individuals are treated as if they cannot change their group membership, even if the group is artificially constructed.
• Argues that the legal system should protect individuals from discrimination based on such artificial but functionally immutable categories.
• Uses comparative legal analysis to show that existing anti-discrimination frameworks can be adapted to include algorithmic groups through reinterpretation of core principles like equal protection and non-discrimination.

Experimental results

Research questions

• RQ1 Can existing anti-discrimination laws in North America and Europe protect individuals from discrimination based on algorithmic groups that are not formally recognized as protected characteristics?
• RQ2 In what ways do algorithmic groups function as de facto immutable characteristics despite being artificially constructed by AI systems?
• RQ3 How do the social and structural consequences of algorithmic group membership mirror the harms associated with legally protected immutable traits?
• RQ4 What legal theory can bridge the gap between the formal absence of algorithmic groups in non-discrimination law and their real-world impact on autonomy and opportunity?
• RQ5 To what extent can the concept of 'artificial immutability' be used to extend legal protection to individuals harmed by AI-driven profiling?

Key findings

• Algorithmic groups—such as those derived from predictive analytics in credit scoring or hiring—often function as if they were immutable, even though they are not formally recognized as such under anti-discrimination law.
• Individuals subjected to algorithmic profiling based on these groups face significant barriers to mobility and opportunity, similar to those experienced by members of legally protected groups.
• The theory of artificial immutability provides a functional basis for legal protection by focusing on the real-world impact of group membership rather than its legal classification.
• Legal systems in North America and Europe currently lack doctrinal tools to address harms from algorithmic groups, despite their substantial social consequences.
• The theory enables reinterpretation of existing anti-discrimination principles to include algorithmic groups without requiring legislative change.
• The framework demonstrates that the core harm of discrimination lies not in the legal status of a characteristic, but in its practical, irreversible impact on individuals’ lives.