[Paper Review] A Decade's Battle on Dataset Bias: Are We There Yet?

Zhuang Liu, Kaiming He|arXiv (Cornell University)|Mar 13, 2024

Artificial Intelligence in Healthcare8 citations

TL;DR

The paper shows modern neural networks can accurately classify which of several large, diverse datasets an image comes from, demonstrating persistent dataset bias even in large-scale, less curated data.

ABSTRACT

We revisit the "dataset classification" experiment suggested by Torralba & Efros (2011) a decade ago, in the new era with large-scale, diverse, and hopefully less biased datasets as well as more capable neural network architectures. Surprisingly, we observe that modern neural networks can achieve excellent accuracy in classifying which dataset an image is from: e.g., we report 84.7% accuracy on held-out validation data for the three-way classification problem consisting of the YFCC, CC, and DataComp datasets. Our further experiments show that such a dataset classifier could learn semantic features that are generalizable and transferable, which cannot be explained by memorization. We hope our discovery will inspire the community to rethink issues involving dataset bias.

Motivation & Objective

• Stimulate rethinking of dataset bias versus model capability in modern architectures.
• Empirically assess whether current datasets retain distinguishable source signatures that models can exploit.
• Investigate whether learned dataset-discriminative features transfer to downstream semantic tasks.
• Examine how data size, augmentation, model architecture, and self-supervised pretraining affect dataset classification performance.

Proposed method

• Define and operationalize a dataset classification task by treating each dataset as a class in an N-way image classification problem.
• Assemble diverse, large-scale datasets (YFCC, CC, DataComp, WIT, LAION, ImageNet) and train ConvNeXt and other architectures on 1M per dataset.
• Evaluate dataset classification accuracy on held-out validation images across dataset combinations.
• Test robustness to low-level artifacts via corruption experiments (color jitter, noise, blur, low res).
• Contrast fully supervised training with pseudo-dataset setups to distinguish memorization from generalization.
• Explore self-supervised pretraining (MAE) followed by linear probing to assess transfer of dataset-discriminative features.

Experimental results

Research questions

• RQ1 Can modern neural networks reliably identify the source dataset of an image from large-scale, diverse datasets?
• RQ2 Do learned dataset-discriminative features transfer to semantic image classification tasks?
• RQ3 Are high dataset-classification accuracies due to memorization or generalizable patterns?
• RQ4 How do data augmentation, model size, and self-supervised pretraining influence dataset classification performance?

Key findings

• Neural networks achieve high dataset classification accuracy across dataset combinations, e.g., 84.7% on YFCC+CC+DataComp held-out validation.
• Accuracy generally rises with more training data and stronger augmentation, suggesting generalizable patterns rather than memorization.
• Even small models can achieve strong dataset classification performance (e.g., ConvNeXt Tiny with 27M params reaches 84.7%).
• Pseudo-dataset experiments indicate memorization is unlikely the main driver in real dataset classification tasks; models fail to generalize in pseudo setups.
• Self-supervised pretraining with linear probing yields substantial transfer for dataset classification (up to 78.4%), and dataset-discriminative features offer nontrivial improvements on ImageNet via linear probing.
• Features learned from dataset classification can provide nontrivial but inferior gains compared to specialized self-supervised methods for semantic tasks.