[Paper Review] On Feature Learning in the Presence of Spurious Correlations

Pavel Izmailov, Polina Kirichenko+2 more|arXiv (Cornell University)|Oct 20, 2022

Industrial Vision Systems and Defect Detection23 citations

TL;DR

The paper shows that standard ERM learned features are often sufficient for strong worst-group performance after retraining only the last layer (DFR), and that improvements from group robustness methods largely come from better last-layer weighting rather than better core feature representations.

ABSTRACT

Deep classifiers are known to rely on spurious features $\unicode{x2013}$ patterns which are correlated with the target on the training data but not inherently relevant to the learning problem, such as the image backgrounds when classifying the foregrounds. In this paper we evaluate the amount of information about the core (non-spurious) features that can be decoded from the representations learned by standard empirical risk minimization (ERM) and specialized group robustness training. Following recent work on Deep Feature Reweighting (DFR), we evaluate the feature representations by re-training the last layer of the model on a held-out set where the spurious correlation is broken. On multiple vision and NLP problems, we show that the features learned by simple ERM are highly competitive with the features learned by specialized group robustness methods targeted at reducing the effect of spurious correlations. Moreover, we show that the quality of learned feature representations is greatly affected by the design decisions beyond the training method, such as the model architecture and pre-training strategy. On the other hand, we find that strong regularization is not necessary for learning high quality feature representations. Finally, using insights from our analysis, we significantly improve upon the best results reported in the literature on the popular Waterbirds, CelebA hair color prediction and WILDS-FMOW problems, achieving 97%, 92% and 50% worst-group accuracies, respectively.

Motivation & Objective

• Investigate how much core (non-spurious) information is decodable from representations learned under spurious correlations.
• Assess whether last-layer retraining (DFR) bridges performance gaps between ERM and group robustness methods.
• Determine how model architecture, pretraining strategy, and regularization influence feature learning in the presence of spurious correlations.
• Evaluate whether improvements from group DRO and related methods stem from better feature representations or from reweighting of features.

Proposed method

• Use Deep Feature Reweighting (DFR) to evaluate core feature decodability by retraining only the last layer on a held-out set where spurious correlations are broken.
• Compare ERM, reweighting (RWG/RWY), and group DRO across multiple vision and NLP tasks.
• Analyze the effect of architecture, pretraining, and regularization on the quality of learned features.
• Measure worst-group accuracy (WGA) and DFR WGA to distinguish base model performance from feature quality.
• Assess spurious vs. core feature decodability via s-DFR (predicting spurious attribute) as a complementary evaluation.

Experimental results

Research questions

• RQ1 How decodable are core (non-spurious) features from representations learned under spurious correlations?
• RQ2 Do group robustness methods improve feature representations, or mainly weight the last-layer features to boost worst-group performance?
• RQ3 How do architecture, pretraining, and regularization influence the quality of learned features under spurious correlations?
• RQ4 Can last-layer retraining (DFR) recover state-of-the-art performance on spurious-benchmark problems without full-model retraining?

Key findings

• DFR applied to ERM features yields competitive or nearly identical worst-group performance to group robustness methods across several benchmarks.
• Improvements of group DRO and related methods are largely due to better weighting in the last linear layer rather than learning superior core features.
• Stronger in-distribution (mean) accuracy generally correlates with better worst-group performance after applying DFR, across architectures and pretraining regimes.
• Pretraining strategy significantly impacts feature quality; strong regularization alone does not consistently improve representations.
• Finetuning on target data provides limited gains when strong pretraining is used, and large-scale pretraining can achieve high DFR WGA even without target-domain finetuning (e.g., Waterbirds with ImageNet22k-pretrained ConvNext).
• On Waterbirds, CelebA, and WILDS-FMOW, DFR with strong pretrained models achieves new high WGA scores (e.g., 97.2% on Waterbirds, 92.2% on CelebA).