[Paper Review] Lipstick on a Pig: Debiasing Methods Cover up Systematic Gender Biases in Word Embeddings But do not Remove Them

Hila Gonen, Yoav Goldberg|arXiv (Cornell University)|Mar 9, 2019

Topic ModelingComputer Science8 references232 citations

TL;DR

The paper shows that popular debiasing methods reduce the gender direction in word embeddings but do not remove underlying gender bias, which remains reflected in neighborhood structure and relative word similarities.

ABSTRACT

Word embeddings are widely used in NLP for a vast range of tasks. It was shown that word embeddings derived from text corpora reflect gender biases in society. This phenomenon is pervasive and consistent across different word embedding models, causing serious concern. Several recent works tackle this problem, and propose methods for significantly reducing this gender bias in word embeddings, demonstrating convincing results. However, we argue that this removal is superficial. While the bias is indeed substantially reduced according to the provided bias definition, the actual effect is mostly hiding the bias, not removing it. The gender bias information is still reflected in the distances between "gender-neutralized" words in the debiased embeddings, and can be recovered from them. We present a series of experiments to support this claim, for two debiasing methods. We conclude that existing bias removal techniques are insufficient, and should not be trusted for providing gender-neutral modeling.

Motivation & Objective

• Motivate and quantify the inadequacy of debiasing methods that target only the gender projection in embeddings.
• Demonstrate that remaining bias correlates with neighbor structure and global geometry of embeddings.
• Provide evidence that current debiasing approaches do not yield truly gender-neutral representations.

Proposed method

• Compare hard-debiased (Bolukbasi et al. 2016b) and GN-GloVe (Zhao et al. 2018) embeddings to their biased counterparts.
• Quantify word bias via projection on the gender direction (he–she).
• Use clustering, neighbor analysis, and WEAT-based associations to assess residual bias.
• Evaluate how well a trained classifier can generalize gender from biased vs debiased word sets.

Experimental results

Research questions

• RQ1 Does debiasing reduce the gender projection of words as defined by the gender direction?
• RQ2 Is residual bias still reflected in word neighborhoods and semantic associations after debiasing?
• RQ3 Can implicit gender information be recovered from debiased embeddings through neighbor analysis or classifiers?

Key findings

• Hard-Debiased words cluster by gender with 92.5% accuracy vs 99.9% in the biased version.
• GN-GloVe shows 85.6% clustering accuracy vs 100% in the biased version.
• Residual bias is evident in the nearest-neighbor structure: words remain close to socially-biased terms after debiasing.
• Correlation between original bias and neighbor-based bias remains significant after debiasing (Pearson r = 0.686 for Hard-Debiased; r = 0.736 for GN-GloVe).
• Profession-related bias shows a strong relation between original bias and number of male neighbors after debiasing (r = 0.606 for Hard-Debiased; r = 0.792 for GN-GloVe).
• Association tests from Caliskan et al. (2017) yield significant effects after debiasing (p-values: Hard-Debiased: 0, 0.00016, 0.0467; GN-GloVe: 7.7e-5, 0.00031, 0.0064).
• Classifiers trained to predict gender from biased words perform poorly after debiasing (Hard-Debiased 88.88% vs 98.25% non-debiased; GN-GloVe 96.53% vs 98.65%).