[Paper Review] QueerBench: Quantifying Discrimination in Language Models Toward Queer Identities

Mae Sosto, Alberto Barrón‐Cedeño|arXiv (Cornell University)|Jun 18, 2024

Gender Studies in Language4 citations

TL;DR

This paper introduces QueerBench, a novel framework that quantifies anti-LGBTQIA+ bias in English large language models (LLMs) using a template-based approach and masked language modeling. It reveals that LLMs generate harmful completions 16.9% of the time when queer identities are subjects—7.2% higher than for non-queer subjects—highlighting systemic discrimination toward queer identities in NLP systems.

ABSTRACT

With the increasing role of Natural Language Processing (NLP) in various applications, challenges concerning bias and stereotype perpetuation are accentuated, which often leads to hate speech and harm. Despite existing studies on sexism and misogyny, issues like homophobia and transphobia remain underexplored and often adopt binary perspectives, putting the safety of LGBTQIA+ individuals at high risk in online spaces. In this paper, we assess the potential harm caused by sentence completions generated by English large language models (LLMs) concerning LGBTQIA+ individuals. This is achieved using QueerBench, our new assessment framework, which employs a template-based approach and a Masked Language Modeling (MLM) task. The analysis indicates that large language models tend to exhibit discriminatory behaviour more frequently towards individuals within the LGBTQIA+ community, reaching a difference gap of 7.2% in the QueerBench score of harmfulness.

Motivation & Objective

• To address the underexplored issue of homophobia and transphobia in NLP, particularly in large language models (LLMs).
• To develop a systematic method for quantifying harmful language generation toward LGBTQIA+ identities, especially beyond binary gender norms.
• To expose and measure discriminatory tendencies in LLMs when generating completions for queer versus non-queer subjects.
• To promote inclusive NLP by identifying biases that marginalize gender-diverse and queer individuals in AI-generated text.
• To release open-source code and data to support future research on anti-LGBTQIA+ bias in NLP.

Proposed method

• Design a template-based prompting framework to elicit sentence completions from LLMs using standardized, neutral contexts.
• Employ a masked language modeling (MLM) task to assess the likelihood of harmful tokens being predicted in response to queer and non-queer subjects.
• Use a curated set of pronouns (binary, neo-pronouns, neutral) and nouns (queer-identifying vs. non-queer) as subjects to evaluate model behavior.
• Calculate the QueerBench score as the percentage of harmful completions across multiple model generations (top-1 and top-5 predictions).
• Compare harmfulness scores across different pronoun and noun categories to detect bias imbalances.
• Analyze model behavior across multiple LLMs (e.g., BERTweet, RoBERTa, ALBERT) in both base and large variants to assess scale effects.

Experimental results

Research questions

• RQ1 How do English LLMs differ in generating harmful completions when the subject is a queer-identifying noun versus a non-queer noun?
• RQ2 What is the level of bias in LLMs when generating completions for binary pronouns, neo-pronouns, and neutral pronouns as subjects?
• RQ3 Does model size (base vs. large) significantly affect the harmfulness of completions toward LGBTQIA+ identities?
• RQ4 To what extent do LLMs exhibit discriminatory behavior toward queer identities compared to non-queer identities in masked language modeling tasks?
• RQ5 How does the QueerBench framework detect and quantify anti-LGBTQIA+ bias in a reproducible and scalable way?

Key findings

• LLMs exhibit significantly higher harmfulness when generating completions for queer subjects, with an average QueerBench score of 16.9%, compared to 9.2% for non-queer subjects.
• The difference in harmfulness between queer and non-queer subjects reaches a peak gap of 7.2% in QueerBench scores, indicating measurable discrimination.
• For pronoun-based subjects, harmfulness is highest for binary pronouns (6.1%), followed by neo-pronouns (5.4%), and lowest for neutral pronouns (4.9%).
• Top-5 predictions show a 13% increase in harmfulness compared to top-1 predictions for noun subjects, suggesting that model diversity amplifies bias.
• The BERTweet base model exhibits disproportionately high harmfulness (27% for queer subjects in top-5), suggesting data or architecture-specific bias.
• Large models show slightly lower overall harmfulness than base models (e.g., 8.2% vs. 11.4% for queer nouns), but the difference is not statistically significant, indicating that scale alone does not reduce bias.