[论文解读] To which reference class do you belong? Measuring racial fairness of reference classes with normative modeling

Saige Rutherford, Thomas Wolfers+5 more|arXiv (Cornell University)|Jul 26, 2024

School Choice and Performance被引用 8

一句话总结

本文研究在脑结构的规范模型中的种族公平性，展示经过预训练且对种族不敏感的模型存在种族偏见，以及包含种族信息可以降低偏差，而来自偏差分数仍可预测种族。强调参考类的代表性以及对偏差的谨慎解读。

ABSTRACT

Reference classes in healthcare establish healthy norms, such as pediatric growth charts of height and weight, and are used to chart deviations from these norms which represent potential clinical risk. How the demographics of the reference class influence clinical interpretation of deviations is unknown. Using normative modeling, a method for building reference classes, we evaluate the fairness (racial bias) in reference models of structural brain images that are widely used in psychiatry and neurology. We test whether including race in the model creates fairer models. We predict self-reported race using the deviation scores from three different reference class normative models, to better understand bias in an integrated, multivariate sense. Across all of these tasks, we uncover racial disparities that are not easily addressed with existing data or commonly used modeling techniques. Our work suggests that deviations from the norm could be due to demographic mismatch with the reference class, and assigning clinical meaning to these deviations should be done with caution. Our approach also suggests that acquiring more representative samples is an urgent research priority.

研究动机与目标

• 量化现有皮层厚度的预训练规范模型中的种族偏见。
• 评估在规范模型中将种族作为预测变量的影响。
• 在两个大型数据集（HCP、UKB）比较三种规范模型配置。
• 确定偏差分数是否揭示种族特定偏差，以及是否可以从偏差预测种族。

提出的方法

• 用 Freesurfer Destrieux 体板区的皮层厚度拟合三种贝叶斯规范模型（预训练、未包含种族、包含种族）。
• 对年龄效应使用 B-spline 基底展开，并通过似然扭曲将非高斯响应映射到高斯潜在空间。
• 对每个区域和受试者计算偏差分数 Z_nd 和残差误差 E_nd。
• 按种族定性总结平均偏差和极端偏差。
• 使用纠正的 FDR 的 t 检验定量测试偏差和残差误差的组间差异。
• 使用带惩罚的逻辑回归，80/20 训练/测试划分和5折交叉验证，从偏差分数预测自我报告的种族。

Figure 1: Overview of analysis workflow. A) Normative models of brain structure were used to generate deviation scores. Three normative models were fit (pre-trained, race not included, and race included) representing two different reference classes and two sets of covariates. B) Normative models wer

实验结果

研究问题

• RQ1 当种族数据未知时，预训练的规范模型中是否存在种族偏见？
• RQ2 在规范模型中包含自我报告的种族是否会减少偏差分数和残差误差中的种族偏见？
• RQ3 来自规范模型的偏差分数是否能在多变量设定中预测自我报告的种族？
• RQ4 基于种族的建模如何影响脑区域和数据集（HCP、UKB）层面的公平性？

主要发现

数据集	组	指标	预训练	未包含种族	包含种族
HCP	W vs. A	偏差	40%	49%	9%
HCP	W vs. B	偏差	55%	51%	5%
UKB	W vs. A	偏差	17%	25%	19%
UKB	W vs. B	偏差	45%	37%	7%
HCP	W vs. A	误差	74%	64%	55%
HCP	W vs. B	误差	56%	28%	54%
UKB	W vs. A	误差	71%	56%	53%
UKB	W vs. B	误差	87%	73%	51%

• 在规范模型中存在种族偏见，白人群体往往围绕零居中，而亚洲/黑人群体则显示对皮层厚度的过高估计或低估，取决于模型类型。
• 在规范模型中包含种族在很大程度上降低了平均偏差的组间差异，尽管某些群体仍存在某些区域的低估。
• 定量测试在残差误差和偏差分数上显示跨模型显著的组间差异，而包含种族的模型通常显示更少的差异。
• 在 UKB，白人组通常显示更大的残差误差，可能是由于样本量不平衡；在 HCP，一些组的残差误差中出现半球侧化效应。
• 在各模型中，偏差和残差误差随种族而异，表明与参考类别的人口统计不匹配可能驱动偏差的临床解读。
• 可以从偏差分数识别出种族，具有显著的准确性，表明种族信息已泄漏到规范模型输出中。

Figure 2: Summary of normative model deviation scores across all three reference classes (pre-trained, race not included, and race included) in HCP and UKB datasets. A) Average (mean) deviations for all brain regions within all racial groups (columns). B) Percentage of extreme deviations (positive a