[Paper Review] Generative Discrimination: What Happens When Generative AI Exhibits Bias, and What Can Be Done About It

Philipp Hacker|arXiv (Cornell University)|Jan 1, 2024

Impact of AI and Big Data on Business and Society5 citations

TL;DR

The paper discusses how generative AI can exhibit discrimination, categorizes outputs, and proposes regulatory updates and accountability to mitigate bias.

ABSTRACT

As generative Artificial Intelligence (genAI) technologies proliferate across sectors, they offer significant benefits but also risk exacerbating discrimination. This chapter explores how genAI intersects with non-discrimination laws, identifying shortcomings and suggesting improvements. It highlights two main types of discriminatory outputs: (i) demeaning and abusive content and (ii) subtler biases due to inadequate representation of protected groups, which may not be overtly discriminatory in individual cases but have cumulative discriminatory effects. For example, genAI systems may predominantly depict white men when asked for images of people in important jobs. This chapter examines these issues, categorizing problematic outputs into three legal categories: discriminatory content; harassment; and legally hard cases like unbalanced content, harmful stereotypes or misclassification. It argues for holding genAI providers and deployers liable for discriminatory outputs and highlights the inadequacy of traditional legal frameworks to address genAI-specific issues. The chapter suggests updating EU laws, including the AI Act, to mitigate biases in training and input data, mandating testing and auditing, and evolving legislation to enforce standards for bias mitigation and inclusivity as technology advances.

Motivation & Objective

• Examine how genAI interacts with non-discrimination laws and identify gaps.
• Identify two main discriminatory output types: demeaning content and representation biases.
• Propose accountability for providers and deployers of genAI and discuss legal framework inadequacies.

Proposed method

• Classify problematic genAI outputs into discriminatory content, harassment, and hard legal cases like unbalanced content or misclassification.
• Discuss liability implications for providers and deployers of genAI outputs.
• Review and suggest updates to EU law, including the AI Act, to address bias in data and model outputs.
• Advocate for mandatory testing, auditing, and standards for bias mitigation and inclusivity.

Experimental results

Research questions

• RQ1 What discriminatory outputs can genAI produce, and how should they be categorized legally?
• RQ2 How should liability be assigned between genAI providers and deployers for discriminatory outputs?
• RQ3 What legal framework changes are needed to address genAI-specific bias and inclusivity?
• RQ4 How can testing and auditing be mandated to mitigate training and input data biases?
• RQ5 What role should EU regulation (e.g., AI Act) play in enforcing bias-mitigation standards?

Key findings

• GenAI outputs can be discriminatory content, harassment, or fall into hard legal categories like unbalanced content and harmful stereotypes.
• There is inadequate alignment between current non-discrimination laws and genAI-specific biases.
• Liability should extend to providers and deployers of genAI for discriminatory outputs.
• EU law, including the AI Act, requires updates to mitigate training and input data biases and to mandate testing and auditing.
• Legislation should evolve to enforce standards for bias mitigation and inclusivity as technology advances.