[Paper Review] Quantifying Inequality in Underreported Medical Conditions.

Divya Shanmugam, Emma Pierson|arXiv (Cornell University)|Oct 8, 2021

Machine Learning and Data Classification31 references4 citations

TL;DR

This paper proposes a novel method for estimating relative prevalence of underreported medical conditions using positive unlabeled learning under the covariate shift assumption, enabling accurate comparison across demographic groups without requiring absolute prevalence estimates. The approach outperforms baselines in synthetic and real-world health data, demonstrating robustness to mild violations of the covariate shift assumption.

ABSTRACT

Estimating the prevalence of a medical condition, or the proportion of the population in which it occurs, is a fundamental problem in healthcare and public health. Accurate estimates of the relative prevalence across groups -- capturing, for example, that a condition affects women more frequently than men -- facilitate effective and equitable health policy which prioritizes groups who are disproportionately affected by a condition. However, it is difficult to estimate relative prevalence when a medical condition is underreported. In this work, we provide a method for accurately estimating the relative prevalence of underreported medical conditions, building upon the positive unlabeled learning framework. We show that under the commonly made covariate shift assumption -- i.e., that the probability of having a disease conditional on symptoms remains constant across groups -- we can recover the relative prevalence, even without restrictive assumptions commonly made in positive unlabeled learning and even if it is impossible to recover the absolute prevalence. We provide a suite of experiments on synthetic and real health data that demonstrate our method's ability to recover the relative prevalence more accurately than do baselines, and the method's robustness to plausible violations of the covariate shift assumption.

Motivation & Objective

• To address the challenge of estimating relative prevalence of underreported medical conditions when standard reporting is biased or incomplete.
• To develop a method that does not require absolute prevalence estimation, which is often infeasible in underreported conditions.
• To enable fairer health policy by accurately identifying groups disproportionately affected by a condition.
• To operate under minimal assumptions, particularly avoiding restrictive conditions common in traditional positive unlabeled learning.
• To ensure robustness to plausible violations of the covariate shift assumption in real-world health data.

Proposed method

• The method leverages the positive unlabeled learning framework, treating diagnosed cases as positive and undiagnosed cases as unlabeled.
• It assumes covariate shift—i.e., the conditional probability of disease given symptoms is constant across groups—enabling relative prevalence estimation.
• The approach uses a reweighting strategy to adjust for group-specific reporting biases, enabling estimation of relative prevalence ratios.
• It formulates the estimation problem as a constrained optimization task to minimize distributional divergence between groups.
• The method does not require estimating absolute prevalence, focusing instead on relative proportions across groups.
• It is designed to be robust to moderate violations of the covariate shift assumption through empirical risk minimization with calibrated weights.

Experimental results

Research questions

• RQ1 Can we accurately estimate the relative prevalence of underreported medical conditions across demographic groups without assuming absolute prevalence?
• RQ2 How does the proposed method perform compared to existing positive unlabeled learning baselines in estimating relative prevalence?
• RQ3 To what extent is the method robust to violations of the covariate shift assumption in real-world health data?
• RQ4 Can the method identify groups with higher relative burden of disease even when diagnosis is systematically underreported in certain populations?
• RQ5 What are the key assumptions required for the method to recover relative prevalence, and how do they compare to those in prior work?

Key findings

• The proposed method achieves significantly more accurate relative prevalence estimates than baseline methods on both synthetic and real health datasets.
• The method successfully recovers relative prevalence ratios even when absolute prevalence cannot be estimated due to underreporting.
• It demonstrates robust performance under plausible violations of the covariate shift assumption, maintaining accuracy in realistic data scenarios.
• The approach outperforms existing positive unlabeled learning methods that rely on restrictive assumptions such as uniform prior or known class imbalance.
• Experiments show that the method can reliably identify groups with higher relative disease burden, even in the absence of complete diagnostic records.