[Paper Review] Echoes of Biases: How Stigmatizing Language Affects AI Performance

Yizhi Liu, Weiguang Wang+2 more|arXiv (Cornell University)|May 17, 2023

Machine Learning in Healthcare4 citations

TL;DR

This study investigates how stigmatizing language (SL) in electronic health records (EHRs) degrades AI performance in mortality prediction, particularly for Black patients. Using a Transformer-based model and explainable AI, it identifies central clinicians in a collaborative network as key drivers of racial disparities and demonstrates that removing SL from these clinicians is more effective for bias mitigation than removing all SL across the corpus.

ABSTRACT

Electronic health records (EHRs) serve as an essential data source for the envisioned artificial intelligence (AI)-driven transformation in healthcare. However, clinician biases reflected in EHR notes can lead to AI models inheriting and amplifying these biases, perpetuating health disparities. This study investigates the impact of stigmatizing language (SL) in EHR notes on mortality prediction using a Transformer-based deep learning model and explainable AI (XAI) techniques. Our findings demonstrate that SL written by clinicians adversely affects AI performance, particularly so for black patients, highlighting SL as a source of racial disparity in AI model development. To explore an operationally efficient way to mitigate SL's impact, we investigate patterns in the generation of SL through a clinicians' collaborative network, identifying central clinicians as having a stronger impact on racial disparity in the AI model. We find that removing SL written by central clinicians is a more efficient bias reduction strategy than eliminating all SL in the entire corpus of data. This study provides actionable insights for responsible AI development and contributes to understanding clinician behavior and EHR note writing in healthcare.

Motivation & Objective

• To investigate the impact of stigmatizing language (SL) in EHR notes on AI-driven mortality prediction performance.
• To identify how SL contributes to racial disparities in AI model outcomes.
• To explore operationally efficient strategies for reducing bias by analyzing clinician influence in SL generation.
• To evaluate the effectiveness of targeting central clinicians in SL mitigation versus global SL removal.

Proposed method

• A Transformer-based deep learning model is trained on EHR notes to predict in-hospital mortality.
• Explainable AI (XAI) techniques are applied to interpret model predictions and assess feature importance.
• A clinician collaborative network is constructed to identify central clinicians based on SL contribution patterns.
• SL is systematically removed from the training data, first globally and then selectively from central clinicians, to compare mitigation efficiency.
• Model performance is evaluated using metrics like AUC-ROC, with subgroup analysis by race to assess disparity.
• Network centrality metrics (e.g., degree, betweenness) are used to quantify clinician influence on SL propagation.

Experimental results

Research questions

• RQ1 How does stigmatizing language in EHR notes affect the performance of AI models in predicting patient mortality?
• RQ2 To what extent does stigmatizing language contribute to racial disparities in AI-driven mortality prediction?
• RQ3 Which clinicians are most influential in generating stigmatizing language, and how does their role affect model bias?
• RQ4 Is removing SL from central clinicians more effective for bias mitigation than removing all SL in the dataset?

Key findings

• Stigmatizing language in EHR notes significantly degrades AI model performance, especially for Black patients, with a notable drop in AUC-ROC for this subgroup.
• Central clinicians—those with high network centrality—generate a disproportionate share of stigmatizing language and are strongly linked to increased racial disparities in model predictions.
• Removing SL written by central clinicians reduces racial disparity more effectively than removing all SL across the entire dataset.
• The model's performance improves substantially when SL from central clinicians is removed, indicating their outsized influence on bias propagation.
• Explainable AI analysis confirms that SL features are disproportionately weighted in predictions for Black patients, reinforcing existing disparities.
• The study demonstrates that targeted intervention on central clinicians offers a more efficient and scalable strategy for bias mitigation than broad data curation.