[论文解读] Oxford Handbook on AI Ethics Book Chapter on Race and Gender

Timnit Gebru|arXiv (Cornell University)|Aug 8, 2019

Ethics and Social Impacts of AI参考文献 8被引用 31

一句话总结

本章由蒂姆尼特·盖布里著述，探讨了人工智能中的系统性种族与性别偏见，指出面部识别和再犯预测工具等AI系统对边缘化群体造成了不成比例的伤害。基于实证研究与真实案例，该文主张采取多维度的AI伦理策略，包括多元化开发团队、监管标准以及历史背景，以减轻伤害。

ABSTRACT

From massive face-recognition-based surveillance and machine-learning-based decision systems predicting crime recidivism rates, to the move towards automated health diagnostic systems, artificial intelligence (AI) is being used in scenarios that have serious consequences in people's lives. However, this rapid permeation of AI into society has not been accompanied by a thorough investigation of the sociopolitical issues that cause certain groups of people to be harmed rather than advantaged by it. For instance, recent studies have shown that commercial face recognition systems have much higher error rates for dark skinned women while having minimal errors on light skinned men. A 2016 ProPublica investigation uncovered that machine learning based tools that assess crime recidivism rates in the US are biased against African Americans. Other studies show that natural language processing tools trained on newspapers exhibit societal biases (e.g. finishing the analogy "Man is to computer programmer as woman is to X" by homemaker). At the same time, books such as Weapons of Math Destruction and Automated Inequality detail how people in lower socioeconomic classes in the US are subjected to more automated decision making tools than those who are in the upper class. Thus, these tools are most often used on people towards whom they exhibit the most bias. While many technical solutions have been proposed to alleviate bias in machine learning systems, we have to take a holistic and multifaceted approach. This includes standardization bodies determining what types of systems can be used in which scenarios, making sure that automated decision tools are created by people from diverse backgrounds, and understanding the historical and political factors that disadvantage certain groups who are subjected to these tools.

研究动机与目标

• 调查AI系统如何在高风险应用中持续并加剧种族与性别不平等。
• 分析AI工具在现实中表现出显著偏见的案例，例如在面部识别与刑事司法算法中的应用。
• 考察导致边缘化社区遭受过度监控与过度自动化的社会与结构性因素。
• 倡导一种超越技术修复的AI伦理整体方法，涵盖开发团队的多样性与政策监管。
• 强调理解历史与政治背景在构建公平AI系统中的重要性。

提出的方法

• 审查关于不同肤色与性别群体在面部识别错误率方面的实证研究。
• 分析《ProPublica》对COMPAS再犯预测工具种族偏见的调查。
• 审视基于历史报纸文本训练的自然语言处理模型中的偏见。
• 突出不同社会经济阶层在自动化决策中的不平等差异。
• 提出一种涵盖制度、社会与技术干预的系统性AI伦理方法。
• 强调多元化团队与标准化机构在塑造伦理AI部署中的作用。

实验结果

研究问题

• RQ1 商业面部识别系统在不同性别与肤色群体中的表现有何差异？
• RQ2 机器学习工具在刑事司法系统中预测再犯时，其种族偏见程度如何？
• RQ3 自然语言处理模型如何反映并再现社会性别刻板印象？
• RQ4 为何社会经济地位较低的群体更频繁地受到自动化决策系统的对待？
• RQ5 为确保AI系统不会对边缘化社区造成不成比例的伤害，需要哪些制度与结构性变革？

主要发现

• 商业面部识别系统在深色皮肤女性中的错误率显著高于浅色皮肤男性。
• COMPAS再犯预测工具对非裔美国被告的假阳性率高于对白人被告。
• 基于报纸文本训练的自然语言处理模型产生性别刻板印象的类比，例如‘男人之于计算机程序员，正如女人之于家庭主妇’。
• 社会经济地位较低的个体比高社会经济地位者更频繁地被自动化决策系统影响。
• AI系统往往在最易表现出偏见的情境中被部署，表明缺乏公平的监督机制。
• 解决AI偏见的全面方案不仅需要技术修复，还必须包括多元化开发团队、监管标准，以及对历史与政治权力结构的关注。