[论文解读] When More is Less: Incorporating Additional Datasets Can Hurt Performance By Introducing Spurious Correlations

Rhys Compton, Lily Zhang+2 more|arXiv (Cornell University)|Aug 8, 2023

Radiomics and Machine Learning in Medical Imaging被引用 4

一句话总结

本文表明，将外部医学影像数据集加入模型会因疾病标签与医院特异性影像伪影之间的虚假相关性而降低模型性能。在43%的评估设置中，使用两家医院联合数据训练的模型在最差群体准确率上反而低于仅使用单一医院数据训练的模型，尽管其训练分布与测试数据的分布相似性有所提高。

ABSTRACT

In machine learning, incorporating more data is often seen as a reliable strategy for improving model performance; this work challenges that notion by demonstrating that the addition of external datasets in many cases can hurt the resulting model's performance. In a large-scale empirical study across combinations of four different open-source chest x-ray datasets and 9 different labels, we demonstrate that in 43% of settings, a model trained on data from two hospitals has poorer worst group accuracy over both hospitals than a model trained on just a single hospital's data. This surprising result occurs even though the added hospital makes the training distribution more similar to the test distribution. We explain that this phenomenon arises from the spurious correlation that emerges between the disease and hospital, due to hospital-specific image artifacts. We highlight the trade-off one encounters when training on multiple datasets, between the obvious benefit of additional data and insidious cost of the introduced spurious correlation. In some cases, balancing the dataset can remove the spurious correlation and improve performance, but it is not always an effective strategy. We contextualize our results within the literature on spurious correlations to help explain these outcomes. Our experiments underscore the importance of exercising caution when selecting training data for machine learning models, especially in settings where there is a risk of spurious correlations such as with medical imaging. The risks outlined highlight the need for careful data selection and model evaluation in future research and practice.

研究动机与目标

• 探究在医学影像中，引入额外数据集是否始终能提升模型泛化能力。
• 识别因虚假相关性导致数据增加引发性能下降的条件。
• 评估数据平衡技术在缓解此类性能下降方面的有效性。
• 为多源医学人工智能中的数据选择与模型评估提供实用指导。

提出的方法

• 在9种疾病标签和4家医院的胸部X光片数据集上，实证评估单家医院与多家长医院数据集训练的模型。
• 以最差群体准确率为首要评估指标，检测不同医院间性能差异。
• 通过欠采样进行数据平衡，训练模型以减少医院与疾病之间的虚假相关性。
• 分析学习到的表征，评估模型是否在无显式监督的情况下编码了医院特异性信号。
• 应用Puli等人（2022）的理论工具，解释为何数据平衡并不总能获得鲁棒模型。
• 通过不同模型架构和训练配置的对比，隔离数据源混合的影响。

Figure 1: High level overview of experiments. We study changes in worst-group accuracy (bold) when training on additional labelled data from an external hospital.

实验结果

研究问题

• RQ1 在胸部X光片分类中，添加外部医院的数据是否总能提升最差群体准确率？
• RQ2 医院特异性影像伪影在引入疾病标签与数据源之间虚假相关性方面起到何种作用？
• RQ3 通过欠采样实现的数据平衡能否有效消除多源训练导致的性能下降？
• RQ4 在何种条件下，多源训练的泛化能力会劣于单源训练？
• RQ5 即使未显式训练识别医院身份，模型表征如何编码医院身份信息？

主要发现

• 在43%的评估设置中，使用两家医院数据训练的模型，其最差群体准确率低于仅使用单一医院数据训练的模型。
• 即使额外数据使训练分布更接近测试分布，性能下降仍会发生，表明分布相似性本身不足以保证泛化能力。
• 多源数据训练的模型学会了将医院身份作为疾病预测的捷径，其表征能够完美区分训练期间未见过的医院。
• 通过欠采样进行数据平衡在某些情况下改善了性能，但在43%的多源训练导致性能下降的设置中，未能提升最差群体准确率。
• 数据平衡并不能保证优于单源训练的性能，且在某些情况下，其结果甚至劣于完全不引入额外数据集的情况。
• 理论分析证实，当底层数据分布复杂时，数据平衡并非解决虚假相关性问题的保证方法。

Figure 2: Change in worst-group performance after including a second training dataset. We see both improvements in performance as expected, but also many deteriorations. For every element in the legend, the first dataset is the base environment, and the second is the added environment; for instance,