[Paper Review] Gender Bias in Coreference Resolution: Evaluation and Debiasing Methods

Jieyu Zhao, Tianlu Wang+3 more|arXiv (Cornell University)|Apr 18, 2018

Topic ModelingComputer Science20 references95 citations

TL;DR

Introduces WinoBias to measure gender bias in coreference across rule-based, feature-based, and neural systems; shows bias exists and can be mitigated with data augmentation and debiased embeddings with minimal impact on performance.

ABSTRACT

We introduce a new benchmark, WinoBias, for coreference resolution focused on gender bias. Our corpus contains Winograd-schema style sentences with entities corresponding to people referred by their occupation (e.g. the nurse, the doctor, the carpenter). We demonstrate that a rule-based, a feature-rich, and a neural coreference system all link gendered pronouns to pro-stereotypical entities with higher accuracy than anti-stereotypical entities, by an average difference of 21.1 in F1 score. Finally, we demonstrate a data-augmentation approach that, in combination with existing word-embedding debiasing techniques, removes the bias demonstrated by these systems in WinoBias without significantly affecting their performance on existing coreference benchmark datasets. Our dataset and code are available at http://winobias.org.

Motivation & Objective

• Motivate the study of gender bias in coreference resolution and its potential societal impact.
• Create a targeted benchmark (WinoBias) using occupation-based pronouns to probe pro- and anti-stereotypical linking.
• Quantify bias by comparing pro-stereotypical and anti-stereotypical conditioning across systems.
• Propose and evaluate debiasing strategies, including gender-swapping data augmentation and debiased word embeddings, to reduce bias while preserving coreference accuracy.

Proposed method

• Construct WinoBias with two types of Winograd-style templates centered on 40 occupations and gendered pronouns.
• Evaluate three representative coreference systems (rule-based, feature-based, and end-to-end neural) on OntoNotes and WinoBias.
• Analyze sources of bias by examining OntoNotes training data, including gender representation and occupation mentions.
• Propose data augmentation by gender swapping to balance gendered correlations in training data.
• Apply debiasing techniques on fixed resources (debiased embeddings, gender lists) and combine with augmentation to reduce bias.
• Show that debiasing reduces the pro-stereotypical vs anti-stereotypical gap on WinoBias with minimal performance loss on OntoNotes.

Experimental results

Research questions

• RQ1 Do coreference systems exhibit gender bias, as measured by differential performance on pro-stereotypical versus anti-stereotypical occupations in WinoBias?
• RQ2 Can data augmentation through gender swapping and embedding/debiasing reduce such bias without substantially harming coreference accuracy on standard benchmarks?
• RQ3 What are the primary sources of bias in commonly used training data (OntoNotes) for coreference tasks?
• RQ4 How do debiasing methods interact with different model architectures (rule-based, feature-based, neural) in mitigating gender bias?

Key findings

• All three representative systems show a measurable disparity between pro-stereotypical and anti-stereotypical conditions on WinoBias.
• A data-augmentation approach (gender swapping) combined with debiased embeddings and resource debiasing reduces or eliminates the bias signal on WinoBias while maintaining competitive performance on OntoNotes.
• Female entities are underrepresented in OntoNotes, and gender swapping can create balanced auxiliary data to counteract this bias.
• Systems can ignore bias when provided with sufficiently strong alternative cues, indicating bias is not unavoidable when cues are robust.
• The proposed data augmentation plus embedding debiasing approach yields little to no loss on key benchmark datasets, suggesting practical utility for debiasing in coreference.