[Paper Review] The Data Addition Dilemma

Judy Hanwen Shen, Inioluwa Deborah Raji+1 more|arXiv (Cornell University)|Aug 8, 2024

Data Quality and ManagementDecision Sciences3 citations

TL;DR

This paper introduces the Data Addition Dilemma, where adding data from diverse medical sources during model scaling can degrade performance due to distribution shift, despite increased data volume. It proposes distribution shift heuristics to guide data source selection, demonstrating that model accuracy can decline when new data introduces harmful statistical shifts, and provides a formal framework to predict when data addition will improve or harm performance in healthcare ML systems.

ABSTRACT

In many machine learning for healthcare tasks, standard datasets are constructed by amassing data across many, often fundamentally dissimilar, sources. But when does adding more data help, and when does it hinder progress on desired model outcomes in real-world settings? We identify this situation as the extit{Data Addition Dilemma}, demonstrating that adding training data in this multi-source scaling context can at times result in reduced overall accuracy, uncertain fairness outcomes, and reduced worst-subgroup performance. We find that this possibly arises from an empirically observed trade-off between model performance improvements due to data scaling and model deterioration from distribution shift. We thus establish baseline strategies for navigating this dilemma, introducing distribution shift heuristics to guide decision-making on which data sources to add in data scaling, in order to yield the expected model performance improvements. We conclude with a discussion of the required considerations for data collection and suggestions for studying data composition and scale in the age of increasingly larger models.

Motivation & Objective

• To formalize the challenge of data scaling in multi-source healthcare machine learning, where increasing dataset size may degrade model performance due to distribution shift.
• To identify when adding new data sources leads to performance deterioration rather than improvement, particularly in real-world clinical settings.
• To develop practical heuristics for data addition that account for distributional changes across sources, ensuring expected performance gains.

Proposed method

• Formalizes the Data Addition Dilemma as a trade-off between performance gains from data scaling and performance loss from distribution shift in multi-source training data.
• Uses f-divergence measures (e.g., Kullback-Leibler divergence) to quantify distributional shifts between training data and test data.
• Derives a theoretical condition using Jensen's inequality and convexity of f-divergences to determine whether adding a new data source will improve or worsen model performance.
• Proposes a heuristic: a new data source should only be added if its divergence from the test distribution exceeds a threshold dependent on the overall training distribution and data composition.
• Empirically validates the heuristic on a real-world ICU dataset from multiple hospitals, comparing model performance across incremental data additions.
• Introduces a framework that balances data scale and data composition, enabling practitioners to make informed decisions about which data sources to include.

Figure 1: Our work introduces the Data Addition Dilemma , where, in a multi-source scaling context, an increase in training dataset size can unexpectedly lead to worse downstream model performance outcomes due to acquired distribution shift.

Experimental results

Research questions

• RQ1 Under what conditions does adding data from additional medical sources lead to reduced model performance despite increased training data size?
• RQ2 How do distribution shifts across heterogeneous data sources affect model accuracy, fairness, and worst-group performance in healthcare ML?
• RQ3 Can we derive a formal criterion to determine whether adding a new data source will improve or degrade model performance in multi-source data scaling?
• RQ4 What role does data composition play in undermining the expected benefits of data scaling in real-world healthcare applications?
• RQ5 How can practitioners make data addition decisions that preserve or improve model performance in the face of distribution shift?

Key findings

• Adding data from new, out-of-distribution sources can reduce overall model accuracy, even when total training data size increases, due to distribution shift.
• The paper identifies a trade-off between performance gains from data scaling and performance degradation from distribution shift, which underlies the Data Addition Dilemma.
• The proposed heuristic based on f-divergence thresholds successfully predicts whether a new data source will improve or harm model performance.
• Model performance on worst subgroups often declines when new data sources are added, even if overall accuracy improves.
• Empirical validation on a multi-hospital ICU dataset shows that data addition decisions significantly impact model outcomes, with some sources causing measurable performance degradation.
• The study demonstrates that data composition changes during scaling are a critical but underappreciated factor in healthcare ML model development.

Figure 2: We illustrate the Data Addition Dilemma . After hospital 443 has exhausted its in-distribution training data, we can consider adding training data from any of the 11 available hospitals in the eICU dataset. The addition of any available hospital data source can lead to uncertain outcomes a