[Paper Review] Reevaluating the Role of Race and Ethnicity in Diabetes Screening

Madison Coots, Soroush Saghafian+2 more|arXiv (Cornell University)|Jun 17, 2023

Healthcare Policy and ManagementEconomics, Econometrics and Finance3 citations

TL;DR

This study reevaluates the clinical utility of including race and ethnicity in diabetes risk prediction models, showing that while such models significantly improve statistical accuracy—particularly for Asian, Black, and Hispanic individuals—only a small fraction of patients receive different screening recommendations, and the resulting clinical benefit is modest due to risks being near the 1.5% decision threshold. The findings suggest that despite gains in prediction accuracy, the real-world impact on screening decisions and health outcomes is limited.

ABSTRACT

There is active debate over whether to consider patient race and ethnicity when estimating disease risk. By accounting for race and ethnicity, it is possible to improve the accuracy of risk predictions, but there is concern that their use may encourage a racialized view of medicine. In diabetes risk models, despite substantial gains in statistical accuracy from using race and ethnicity, the gains in clinical utility are surprisingly modest. These modest clinical gains stem from two empirical patterns: first, the vast majority of individuals receive the same screening recommendation regardless of whether race or ethnicity are included in risk models; and second, for those who do receive different screening recommendations, the difference in utility between screening and not screening is relatively small. Our results are based on broad statistical principles, and so are likely to generalize to many other risk-based clinical decisions.

Motivation & Objective

• To assess whether incorporating race and ethnicity into diabetes risk prediction models improves clinical decision-making beyond statistical accuracy.
• To investigate the disconnect between improved statistical performance and actual clinical utility in risk-based screening recommendations.
• To evaluate whether the use of race and ethnicity in risk models leads to meaningful changes in screening decisions for patients at risk of Type 2 diabetes.
• To examine the utility of screening decisions across racial and ethnic groups using a utility-based decision framework grounded in patient-level costs and benefits.
• To determine whether the clinical value of race-aware models justifies their use, especially given concerns about racialization in medicine.

Proposed method

• Utilized data from four NHANES cycles (2011–2018) on 18,000 non-pregnant adults aged 18–70 with BMI 18.5–50.0 kg/m².
• Developed race-unaware risk models using age and BMI to predict diabetes incidence, then compared them to race-aware models that included race and ethnicity as predictors.
• Calibrated risk predictions by race and ethnicity to assess miscalibration—showing that race-unaware models systematically underestimate risk in Asian, Black, and Hispanic individuals.
• Applied a utility-based decision framework to evaluate screening recommendations, using a 1.5% diabetes risk threshold as the decision point for screening.
• Quantified the net utility of screening by comparing expected benefits (early detection) against costs (monetary and non-monetary), assuming $100 per util.
• Evaluated the proportion of patients whose screening recommendation changed when race and ethnicity were included, and assessed the magnitude of utility change for those individuals.

Figure 1: Assessing the statistical and clinical utility of race and ethnicity in diabetes risk estimation. Upper left: Calibration plot for a race-unaware risk model, showing that racial minorities have significantly higher empirical risk of diabetes compared to their race-unaware risk prediction;

Experimental results

Research questions

• RQ1 To what extent does including race and ethnicity in diabetes risk models improve statistical prediction accuracy across racial and ethnic groups?
• RQ2 How many patients receive different screening recommendations when race and ethnicity are included in risk models compared to race-unaware models?
• RQ3 What is the clinical utility of the changed screening recommendations for patients whose risk estimates cross the 1.5% threshold due to inclusion of race and ethnicity?
• RQ4 Why do substantial improvements in statistical accuracy not translate into proportionally larger clinical benefits in screening decisions?
• RQ5 How do the utility gains from race-aware models compare to the risks of reinforcing racialized medicine or reducing trust in healthcare systems?

Key findings

• Race-unaware models significantly miscalibrate diabetes risk: among individuals with a 1% predicted risk, Asian Americans had a 2% actual diabetes prevalence, while White Americans had a 1.5% prevalence.
• Including race and ethnicity in risk models improved statistical accuracy, particularly for Asian Americans, who were substantially underpredicted by race-unaware models.
• Only a small fraction of patients (less than 10%) received different screening recommendations when race and ethnicity were included, despite large improvements in prediction accuracy.
• For patients whose screening recommendation changed, the net utility of screening was relatively small because their predicted risk was close to the 1.5% decision threshold.
• Among Asian Americans with a race-unaware predicted risk below 1.5%, some had positive utility for screening, yet were counseled against it due to the threshold—highlighting missed opportunities.
• The clinical utility gains from race-aware models are modest overall, even though statistical accuracy improves substantially, due to the concentration of decision changes near the point of indifference.

Figure A1: Calibration plot for an extended race-unaware risk model that includes the following covariates in addition to age and BMI: gender, weight, height, waist circumference, self-reported greatest weight, whether a patient’s close family members have diabetes, whether the patient is depressed,