[Paper Review] Interpretable bias mitigation for textual data: Reducing gender bias in patient notes while maintaining classification performance

Joshua R. Minot, Nicholas Cheney+4 more|arXiv (Cornell University)|Mar 10, 2021

Machine Learning in Healthcare61 references4 citations

TL;DR

This paper proposes an interpretable data augmentation method to reduce gender bias in clinical notes by identifying and replacing gendered language using BERT-based classifiers and divergence metrics. It achieves significant bias reduction with minimal performance degradation in health condition classification tasks, demonstrating that debiasing can be both effective and practical in real-world EHR applications.

ABSTRACT

Medical systems in general, and patient treatment decisions and outcomes in particular, are affected by bias based on gender and other demographic elements. As language models are increasingly applied to medicine, there is a growing interest in building algorithmic fairness into processes impacting patient care. Much of the work addressing this question has focused on biases encoded in language models -- statistical estimates of the relationships between concepts derived from distant reading of corpora. Building on this work, we investigate how word choices made by healthcare practitioners and language models interact with regards to bias. We identify and remove gendered language from two clinical-note datasets and describe a new debiasing procedure using BERT-based gender classifiers. We show minimal degradation in health condition classification tasks for low- to medium-levels of bias removal via data augmentation. Finally, we compare the bias semantically encoded in the language models with the bias empirically observed in health records. This work outlines an interpretable approach for using data augmentation to identify and reduce the potential for bias in natural language processing pipelines.

Motivation & Objective

• To address gender bias in electronic health records (EHRs) that may stem from provider language choices and language model tendencies.
• To develop an interpretable method for identifying and reducing gendered language in clinical notes without compromising classification performance.
• To compare empirically observed bias in real EHR data with bias encoded in pre-trained language models like BERT.
• To enable stakeholders to make informed trade-offs between bias reduction and model performance through transparent, human-readable debiasing techniques.
• To explore the impact of debiasing on downstream clinical classification tasks using data augmentation with interpretable trimming and replacement strategies.

Proposed method

• The authors use a rank-turbulence divergence (RTD) metric to quantify differences in word-frequency distributions between male- and female-patient notes, identifying gendered language patterns.
• They train BERT-based gender classifiers to detect gendered terms in clinical text, enabling targeted identification of biased language.
• A data augmentation pipeline applies iterative trimming and replacement of gendered n-grams based on bias rankings derived from RTD and model predictions.
• The method allows users to select different 'trim levels' to control the trade-off between bias reduction and performance degradation.
• The approach is applied to two real-world clinical note datasets, with performance evaluated on health condition classification tasks.
• The framework supports interpretable decision-making by highlighting specific terms and their bias rankings in plain English, enabling clinician and practitioner review.

Experimental results

Research questions

• RQ1 To what extent does gendered language in clinical notes contribute to bias in downstream classification models?
• RQ2 Can interpretable data augmentation techniques reduce gender bias in EHRs without significantly degrading classification performance?
• RQ3 How do empirically observed biases in real clinical notes compare to biases encoded in pre-trained language models like BERT?
• RQ4 What is the impact of different levels of language trimming and replacement on model accuracy and fairness?
• RQ5 Can clinicians and practitioners meaningfully interpret and act on bias rankings derived from divergence and model-based analysis?

Key findings

• The proposed method significantly reduces gender bias in clinical notes using interpretable data augmentation, with minimal degradation in health condition classification performance.
• At low to medium levels of bias removal, the method maintains high classification accuracy, indicating a favorable trade-off between fairness and utility.
• The rank-turbulence divergence metric effectively identifies gendered language patterns, even in non-explicitly gendered terms, by comparing word-frequency distributions across patient groups.
• BERT-based classifiers successfully detect gendered language in clinical text, with bias rankings providing actionable insights for clinicians and developers.
• The study reveals that much of the gender bias in clinical notes stems from terms distinct from medically relevant features, suggesting that bias can be reduced without harming clinical signal detection.
• The approach enables transparent, human-readable decisions on which terms to modify, supporting ethical and accountable AI in healthcare.