[Paper Review] Harms of Gender Exclusivity and Challenges in Non-Binary Representation in Language Technologies

Sunipa Dev, Masoud Monajatipoor+4 more|arXiv (Cornell University)|Aug 27, 2021

Natural Language Processing Techniques38 references4 citations

TL;DR

This paper investigates how gender-exclusive language models perpetuate harm by erasing non-binary identities through biased representations in NLP systems. Using surveys of non-binary individuals and analysis of GloVe and BERT embeddings, it demonstrates that binary gender assumptions in datasets and models lead to misgendering and underrepresentation, with BERT showing separable representations for singular 'they' but still propagating bias through pronoun association tasks.

ABSTRACT

Gender is widely discussed in the context of language tasks and when examining the stereotypes propagated by language models. However, current discussions primarily treat gender as binary, which can perpetuate harms such as the cyclical erasure of non-binary gender identities. These harms are driven by model and dataset biases, which are consequences of the non-recognition and lack of understanding of non-binary genders in society. In this paper, we explain the complexity of gender and language around it, and survey non-binary persons to understand harms associated with the treatment of gender as binary in English language technologies. We also detail how current language representations (e.g., GloVe, BERT) capture and perpetuate these harms and related challenges that need to be acknowledged and addressed for representations to equitably encode gender information.

Motivation & Objective

• To investigate the harms caused by treating gender as binary in language technologies, particularly for non-binary individuals.
• To understand how model and dataset biases perpetuate the erasure and misgendering of non-binary people in NLP systems.
• To analyze how static (GloVe) and contextual (BERT) embeddings encode and propagate non-binary gender representation issues.
• To evaluate the extent to which language models can distinguish between singular 'they' and plural 'they' in context, and whether this distinction leads to misgendering.

Proposed method

• Conducted a survey with non-binary individuals familiar with AI to identify perceived harms in NLP tasks related to gender representation.
• Collected and analyzed a balanced, manually annotated dataset of sentences using singular 'they', 'he', 'she', and plural 'they' for BERT evaluation.
• Used cosine similarity and WEAT scores to measure semantic similarity and bias between gendered pronouns and associated words.
• Applied BERT to masked pronoun prediction tasks to assess whether representations of singular 'they' are distinguishable from plural 'they' and binary pronouns.
• Evaluated misgendering through standardized sentence templates with masked pronouns, measuring model predictions across different gendered pronouns.
• Analyzed embedding spaces using GloVe and BERT to detect disparities in representation quality for non-binary-associated terms.

Experimental results

Research questions

• RQ1 To what extent do current language models like BERT and GloVe misrepresent or fail to represent non-binary pronouns such as 'they' and 'xe'?
• RQ2 How do biases in training data and model architecture lead to the cyclical erasure of non-binary identities in NLP systems?
• RQ3 Can BERT distinguish between singular and plural uses of 'they' in context, and does this affect misgendering outcomes?
• RQ4 What are the specific representational disparities in word embeddings that contribute to non-binary erasure in downstream NLP tasks?
• RQ5 How do non-binary individuals perceive the harms of binary-gendered language technologies in real-world applications?

Key findings

• BERT models show significantly lower similarity between singular 'they' and plural 'they' in representation, indicating some capacity for distinction, but still exhibit bias in downstream tasks.
• The WEAT score for binary pronouns vs. non-binary pronouns was 0.2, indicating a measurable but weak bias, while the score for binary words vs. non-binary proxies reached 0.718, showing strong semantic separation.
• Cosine similarity between 'they' (singular) and 'he'/'she' was 0.395 and 0.390 respectively, while similarity with 'them' was 0.389, suggesting non-binary pronouns are not meaningfully distinguished from plural pronouns.
• Non-binary pronouns like 'xe', 'xem', and 'zir' showed negative or near-zero similarity with binary pronouns, indicating poor integration into existing embedding spaces.
• The BERT classifier achieved 83.3% accuracy in distinguishing singular 'they' from plural 'they', but confusion matrices reveal persistent misclassification, especially when context is ambiguous.
• Survey results revealed that non-binary individuals experience frequent misgendering and erasure in NLP systems, with many reporting that current models fail to recognize or respect their gender identity.