[Paper Review] Unmasking the Mask -- Evaluating Social Biases in Masked Language Models

Masahiro Kaneko, Danushka Bollegala|arXiv (Cornell University)|Apr 15, 2021

Topic Modeling20 citations

TL;DR

This paper proposes All Unmasked Likelihood (AUL) and AUL with Attention weights (AULA), novel bias evaluation measures for masked language models (MLMs) that overcome limitations in prior methods by predicting all unmasked tokens simultaneously, avoiding selection bias from masking and improving reliability. AUL and AULA outperform existing metrics in detecting social biases—especially in gender, race, and occupation stereotypes—by showing higher accuracy and better alignment with human-annotated bias ratings on StereoSet and CrowS-Pairs datasets.

ABSTRACT

Masked Language Models (MLMs) have shown superior performances in numerous downstream NLP tasks when used as text encoders. Unfortunately, MLMs also demonstrate significantly worrying levels of social biases. We show that the previously proposed evaluation metrics for quantifying the social biases in MLMs are problematic due to following reasons: (1) prediction accuracy of the masked tokens itself tend to be low in some MLMs, which raises questions regarding the reliability of the evaluation metrics that use the (pseudo) likelihood of the predicted tokens, and (2) the correlation between the prediction accuracy of the mask and the performance in downstream NLP tasks is not taken into consideration, and (3) high frequency words in the training data are masked more often, introducing noise due to this selection bias in the test cases. To overcome the above-mentioned disfluencies, we propose All Unmasked Likelihood (AUL), a bias evaluation measure that predicts all tokens in a test case given the MLM embedding of the unmasked input. We find that AUL accurately detects different types of biases in MLMs. We also propose AUL with attention weights (AULA) to evaluate tokens based on their importance in a sentence. However, unlike AUL and AULA, previously proposed bias evaluation measures for MLMs systematically overestimate the measured biases, and are heavily influenced by the unmasked tokens in the context.

Motivation & Objective

• Address the unreliability of existing bias evaluation metrics for masked language models (MLMs), which depend on low-accuracy masked token predictions and ignore context-induced biases.
• Overcome selection bias in evaluation caused by frequent word masking, which distorts bias measurement due to skewed token frequency distributions.
• Develop a bias evaluation method that isolates intrinsic model biases from those introduced during fine-tuning or context-dependent prediction.
• Improve alignment between automated bias scores and human-annotated bias ratings by incorporating attention weights to reflect token importance.
• Demonstrate that current evaluation metrics systematically overestimate bias due to flawed assumptions about token independence and low prediction confidence.

Proposed method

• Propose All Unmasked Likelihood (AUL), which uses the MLM to predict all unmasked tokens in a sentence simultaneously, using the full unmasked input embedding to avoid masking-induced distortions.
• Introduce AUL with Attention weights (AULA), which reweight token likelihoods based on self-attention scores to prioritize semantically important words and reduce noise from function words.
• Use the pseudo-likelihood of predicted tokens as a bias score, where lower likelihood for anti-stereotypical sentences indicates higher detected bias.
• Apply AUL and AULA to evaluate MLMs on two benchmark datasets: StereoSet (SS) and CrowS-Pairs (CP), comparing performance against prior metrics like SSS and CPS.
• Calibrate bias detection using binary classification tasks (ROC-AUC) to assess agreement between MLM-based scores and human-annotated bias labels.
• Conduct ablation studies by shuffling sentence tokens or replacing words with unrelated ones to test AUL’s sensitivity to linguistic coherence and meaningful associations.

Experimental results

Research questions

• RQ1 To what extent do existing bias evaluation metrics for MLMs fail due to low masked token prediction accuracy and reliance on pseudo-likelihood in ambiguous contexts?
• RQ2 How does masking frequency bias—favoring high-frequency words—affect the reliability of prior bias evaluation metrics?
• RQ3 Can a bias evaluation method that predicts all unmasked tokens simultaneously reduce noise and improve detection of social biases in MLMs?
• RQ4 How well do AUL and AULA align with human-annotated bias judgments compared to prior metrics on CrowS-Pairs and StereoSet?
• RQ5 To what extent does the attention mechanism improve bias evaluation by weighting more meaningful tokens appropriately?

Key findings

• AUL achieved a 95.71-point accuracy drop on the StereoSet dataset compared to prior pseudo-likelihood-based methods, indicating significantly improved reliability in detecting correct anti-stereotypical responses.
• AULA outperformed all prior metrics in aligning with human bias ratings on CrowS-Pairs, showing the highest agreement via ROC-AUC curves, particularly in detecting subtle biases.
• The proposed method AUL was sensitive to sentence structure and meaning, as demonstrated by significant performance drops when tokens were shuffled or replaced with unrelated words.
• Prior metrics like CPS and SSS systematically overestimated bias due to masking frequency bias and independence assumptions, with CPS incorrectly rating anti-stereotypical sentences as less biased when high-frequency words were masked.
• All evaluated MLMs—including BERT, RoBERTa, and ALBERT—showed concerning levels of social bias, confirming that intrinsic bias is widespread and not limited to specific architectures.
• AUL and AULA demonstrated robustness across different MLMs and bias types, with AULA showing the strongest correlation with human-annotated bias outcomes, especially in complex, context-sensitive cases.