[论文解读] Identification of Bias Against People with Disabilities in Sentiment Analysis and Toxicity Detection Models

Pranav Narayanan Venkit, Shomir Wilson|arXiv (Cornell University)|Nov 25, 2021

Text Readability and Simplification被引用 14

一句话总结

本文介绍了情感中的偏见识别测试（BITS）语料库——一个包含1,126个句子的数据集，旨在检测情感与毒性模型中的残障相关偏见。研究发现，四种情感分析工具（VADER、TextBlob、Google Cloud NLP、DistilBERT）和两种毒性分类器（Toxic Comment Classification 和 Unintended Bias in Toxic Comments）存在统计上显著的负面偏见，p值低至2e-16，表明与残障相关的术语被系统性地错误分类为负面或有毒。

ABSTRACT

Sociodemographic biases are a common problem for natural language processing, affecting the fairness and integrity of its applications. Within sentiment analysis, these biases may undermine sentiment predictions for texts that mention personal attributes that unbiased human readers would consider neutral. Such discrimination can have great consequences in the applications of sentiment analysis both in the public and private sectors. For example, incorrect inferences in applications like online abuse and opinion analysis in social media platforms can lead to unwanted ramifications, such as wrongful censoring, towards certain populations. In this paper, we address the discrimination against people with disabilities, PWD, done by sentiment analysis and toxicity classification models. We provide an examination of sentiment and toxicity analysis models to understand in detail how they discriminate PWD. We present the Bias Identification Test in Sentiments (BITS), a corpus of 1,126 sentences designed to probe sentiment analysis models for biases in disability. We use this corpus to demonstrate statistically significant biases in four widely used sentiment analysis tools (TextBlob, VADER, Google Cloud Natural Language API and DistilBERT) and two toxicity analysis models trained to predict toxic comments on Jigsaw challenges (Toxic comment classification and Unintended Bias in Toxic comments). The results show that all exhibit strong negative biases on sentences that mention disability. We publicly release BITS Corpus for others to identify potential biases against disability in any sentiment analysis tools and also to update the corpus to be used as a test for other sociodemographic variables as well.

研究动机与目标

• 调查并揭示情感与毒性检测模型对残障人士（PWD）的隐性偏见。
• 解决NLP领域中长期被忽视的 ableist（能力歧视）偏见问题，特别是针对在社交媒体数据上训练的模型。
• 开发一种标准化、与模型无关的测试方法，以检测社会人口学偏见，初期聚焦于残障问题。
• 通过在广泛使用的公共工具中实现系统性偏见评估，推动NLP领域的公平性与包容性。
• 将BITS语料库作为公开资源发布，以支持持续的偏见检测，并为未来扩展至其他社会人口群体提供基础。

提出的方法

• 设计一个包含1,126个英文句子的精选语料库，涵盖中性情感与含情感的句子，聚焦于残障相关术语和社会群体指称。
• 采用基于模板的方法构建BITS语料库，生成多样化且上下文受控的句子，以探测潜在偏见。
• 应用该语料库评估四种广泛使用的情感分析模型：VADER、TextBlob、Google Cloud Natural Language API 和 DistilBERT。
• 测试来自Detoxify库的两种毒性检测模型：Toxic Comment Classification 和 Unintended Bias in Toxic Comments。
• 使用包含社会人口学群体指标的线性回归分析，衡量模型输出中偏见的统计显著性。
• 进行p值分析，比较不同社会人口学群体（包括PWD）的情感与毒性评分。

实验结果

研究问题

• RQ1 广泛使用的情感分析模型是否对残障人士表现出统计上显著的偏见？
• RQ2 当分类包含残障相关术语的句子时，毒性检测模型表现如何？
• RQ3 一种标准化、与模型无关的语料库能否有效识别NLP系统中的隐性偏见？
• RQ4 社交媒体训练的模型在多大程度上会将残障相关内容错误分类为负面或有毒？
• RQ5 BITS语料库能否扩展至检测除残障外的其他偏见，如种族、性别或社会经济地位？

主要发现

• 所有四种情感分析模型——VADER、TextBlob、Google Cloud NLP 和 DistilBERT——均对残障相关内容表现出统计上显著的负面偏见，p值低至2e-16。
• Toxic Comment Classification 和 Unintended Bias in Toxic Comments 模型对 'autistic' 和 'mentally handicapped' 等术语均赋予显著更高的毒性评分，表明存在强烈的隐性 ableist 偏见。
• DSBL:S（残障特定偏见）比较的p值在VADER和DistilBERT中均为2e-16，证实了观察到的偏见具有极高的统计显著性。
• 残障相关术语的毒性评分标准差极低，表明其被一致高估为有毒，而非随机噪声。
• 经过专门训练以对PWD保持上下文敏感的 Toxicity_Biased 模型，其表现与原始模型无异，表明若训练数据仍存在偏差，偏见缓解措施可能无效。
• BITS语料库成功在所有测试模型中暴露了偏见，证明其作为可复现、公开的基准，在评估NLP系统中残障相关偏见方面的实用价值。