[Paper Review] Understanding the Origins of Bias in Word Embeddings

Marc-Etienne Brunet, Colleen Alkalay-Houlihan+2 more|arXiv (Cornell University)|Oct 8, 2018

Topic Modeling19 references64 citations

TL;DR

The paper introduces an efficient influence-function-based method to approximate how removing small parts of the training corpus would affect bias in word embeddings, enabling identification of bias-inducing documents without full retraining.

ABSTRACT

The power of machine learning systems not only promises great technical progress, but risks societal harm. As a recent example, researchers have shown that popular word embedding algorithms exhibit stereotypical biases, such as gender bias. The widespread use of these algorithms in machine learning systems, from automated translation services to curriculum vitae scanners, can amplify stereotypes in important contexts. Although methods have been developed to measure these biases and alter word embeddings to mitigate their biased representations, there is a lack of understanding in how word embedding bias depends on the training data. In this work, we develop a technique for understanding the origins of bias in word embeddings. Given a word embedding trained on a corpus, our method identifies how perturbing the corpus will affect the bias of the resulting embedding. This can be used to trace the origins of word embedding bias back to the original training documents. Using our method, one can investigate trends in the bias of the underlying corpus and identify subsets of documents whose removal would most reduce bias. We demonstrate our techniques on both a New York Times and Wikipedia corpus and find that our influence function-based approximations are very accurate.

Motivation & Objective

• Motivate and formalize how bias arises in word embeddings and why understanding its origins matters.
• Introduce a tractable method to estimate the impact of removing individual documents on embedding bias without retraining.
• Apply the method to GloVe on Wikipedia and NYT corpora and validate accuracy against ground-truth retraining results.
• Analyze how influential documents relate to WEAT bias metrics and explore cross-method robustness (e.g., to word2vec).
• Provide qualitative insights into the distribution of bias within training corpora and the limitations of WEAT.

Proposed method

• Formalize differential bias: define the bias change when removing a corpus part p as ΔpB = B(w) − B(ŵ).
• Decompose co-occurrence perturbations by treating the corpus co-occurrence matrix X as the sum of per-document matrices X(k).
• Employ a bias gradient ∇X B(w(X)) to approximate how bias changes with small perturbations in X (via chain rule ∇X B(w(X)) = ∇w B(w) ∇X w(X)).
• Derive an efficient, block-diagonal Hessian structure for GloVe to approximate how each word vector wi changes when removing a document: ŵi ≈ w*i − (1/V) Hwi^−1 [∇wi Li(ṼXi, w) − ∇wi Li(Xi, w)].
• Only compute changes for WEAT-relevant words in a document to obtain Δdoc B ≈ Bweat(w*) − Bweat(ŵ).
• Provide Algorithm 1: Approximating Differential Bias to apply the approximation across all documents without retraining.

Experimental results

Research questions

• RQ1 Can we identify which documents in a training corpus contribute most to a word-embedding bias without retraining the model?
• RQ2 How accurately does the proposed differential-bias approximation predict the actual bias change when removing documents?
• RQ3 Does the method generalize across embedding schemes (GloVe vs. word2vec) and bias metrics (WEAT variants)?
• RQ4 What qualitative patterns emerge about bias distribution and rare words when perturbing the corpus?

Key findings

• The approximation highly correlates with ground-truth bias changes after document removals (r^2 ≥ 0.985 across configurations).
• The targeted perturbations (removing bias-increasing/decreasing documents) reproduce bias changes far more reliably than random perturbations, with only 2 of 40 targeted sets failing significance in Wiki/NYT tests.
• Removing top bias-influencing documents in NYT can substantially alter WEAT1 bias in word2vec (e.g., from 1.35 baseline to 0.11 after removing 10k documents).
• The method outperforms a PPMI-based baseline in predicting bias changes (e.g., 40% reduction vs about 4% for analogous Wiki perturbations).
• Influential documents identified by the method align with intuitive semantic content (e.g., articles about male scientists driving bias) and reveal that rare words are more sensitive to perturbations.
• The analysis exposes limitations and vulnerabilities of WEAT (e.g., sensitivity to rare words) and shows cross-metric effects on other bias measures.