[Paper Review] Bias Amplification in Artificial Intelligence Systems

Kirsten Lloyd|arXiv (Cornell University)|Sep 20, 2018

Ethics and Social Impacts of AI1 references43 citations

TL;DR

The paper argues that AI systems can amplify biases present in training data and emphasizes the need for policy-oriented data standards and inclusive practices to mitigate this risk.

ABSTRACT

As Artificial Intelligence (AI) technologies proliferate, concern has centered around the long-term dangers of job loss or threats of machines causing harm to humans. All of this concern, however, detracts from the more pertinent and already existing threats posed by AI today: its ability to amplify bias found in training datasets, and swiftly impact marginalized populations at scale. Government and public sector institutions have a responsibility to citizens to establish a dialogue with technology developers and release thoughtful policy around data standards to ensure diverse representation in datasets to prevent bias amplification and ensure that AI systems are built with inclusion in mind.

Motivation & Objective

• Highlight the risk that AI can amplify biases in training data and affect marginalized populations at scale.
• Advocate for government and public sector engagement with technologists to address bias amplification.
• Recommend thoughtful policy development around data standards to ensure diverse representation in datasets.
• Promote inclusion-focused AI development practices to prevent bias amplification.

Proposed method

• Policy-oriented analysis grounded in the issue of bias amplification.
• Advocacy for dialogue between government bodies and technology developers.
• Recommendation of data standards and diverse representation as preventative measures.

Experimental results

Research questions

• RQ1 How do AI systems amplify biases found in training datasets?
• RQ2 What policy measures and data standards are effective to prevent bias amplification?
• RQ3 How can public sector engagement foster inclusion in AI systems?

Key findings

• AI technologies can amplify existing biases and impact marginalized populations at scale.
• There is a responsibility for government and public institutions to engage with technology developers on this issue.
• Policy guidance around data standards is needed to ensure diverse representation in training datasets.
• Inclusion-focused practices should be integrated into AI system design to mitigate bias amplification.