[Paper Review] On the Foundations of Shortcut Learning

Katherine L. Hermann, Hossein Mobahi+2 more|arXiv (Cornell University)|Oct 24, 2023

Generative Adversarial Networks and Image Synthesis4 citations

TL;DR

This paper investigates the root causes of shortcut learning in deep neural networks by systematically manipulating both feature predictivity and availability in a controlled generative dataset framework. It demonstrates that nonlinear architectures exhibit inherent shortcut bias due to inductive bias, with deeper models amplifying this effect, and provides theoretical support via Neural Tangent Kernel analysis, showing that shortcut learning is an inevitable property of nonlinear learning systems.

ABSTRACT

Deep-learning models can extract a rich assortment of features from data. Which features a model uses depends not only on \emph{predictivity} -- how reliably a feature indicates training-set labels -- but also on \emph{availability} -- how easily the feature can be extracted from inputs. The literature on shortcut learning has noted examples in which models privilege one feature over another, for example texture over shape and image backgrounds over foreground objects. Here, we test hypotheses about which input properties are more available to a model, and systematically study how predictivity and availability interact to shape models' feature use. We construct a minimal, explicit generative framework for synthesizing classification datasets with two latent features that vary in predictivity and in factors we hypothesize to relate to availability, and we quantify a model's shortcut bias -- its over-reliance on the shortcut (more available, less predictive) feature at the expense of the core (less available, more predictive) feature. We find that linear models are relatively unbiased, but introducing a single hidden layer with ReLU or Tanh units yields a bias. Our empirical findings are consistent with a theoretical account based on Neural Tangent Kernels. Finally, we study how models used in practice trade off predictivity and availability in naturalistic datasets, discovering availability manipulations which increase models' degree of shortcut bias. Taken together, these findings suggest that the propensity to learn shortcut features is a fundamental characteristic of deep nonlinear architectures warranting systematic study given its role in shaping how models solve tasks.

Motivation & Objective

• To understand why deep learning models prefer shortcut features—such as texture or background—over more meaningful, core features like shape.
• To isolate and quantify the roles of predictivity and availability in shaping model feature selection, beyond statistical correlation alone.
• To develop a minimal, explicit generative framework that controls both predictivity and availability of latent features in synthetic datasets.
• To investigate how model architecture, particularly depth and nonlinearity, influences shortcut bias.
• To provide a theoretical foundation for shortcut learning using Neural Tangent Kernel theory, explaining its inevitability in nonlinear models.

Proposed method

• The authors design a generative model that synthesizes binary classification datasets with two latent features: one core (high predictivity, low availability), one shortcut (low predictivity, high availability).
• Availability is quantified through two factors: (1) the singular values of the data-generating transformation, and (2) the degree of nonlinearity in the feature mapping.
• A shortcut bias metric is defined as the deviation of a model’s feature reliance from that of an optimal, Bayes-optimal classifier.
• The study uses linear models, shallow ReLU/Tanh networks, and deeper architectures to test sensitivity to predictivity and availability.
• Theoretical analysis is grounded in Neural Tangent Kernel (NTK) theory to explain why shortcut bias emerges in nonlinear models.
• Empirical validation is extended to naturalistic datasets using availability-altering data augmentations (e.g., texture vs. shape manipulations).

Figure 1: Synthetic data. A: Two datasets differing in the predictivity of $z_{s}$ . B: Schematic of the embedding procedure manipulating availability via the mapping from $\boldsymbol{z}$ to $\boldsymbol{x}$ . Dashed boxes are optional.

Experimental results

Research questions

• RQ1 How do predictivity and availability jointly determine feature selection in deep learning models?
• RQ2 Why do nonlinear models exhibit stronger shortcut bias than linear models, even when features are equally predictive?
• RQ3 To what extent can availability—defined by singular values and nonlinearity—explain anomalous feature preferences beyond statistical predictivity?
• RQ4 Is shortcut bias an inevitable consequence of nonlinear architecture, as suggested by NTK theory?
• RQ5 Can availability manipulations in real images systematically shift model behavior toward or away from shortcut features?

Key findings

• Linear models show minimal shortcut bias, indicating that the bias is not inherent in the learning objective but arises from architectural inductive bias.
• Introducing a single hidden ReLU or Tanh layer induces significant shortcut bias, with deeper networks amplifying this effect.
• Theoretical analysis using Neural Tangent Kernels shows that shortcut bias is an inevitable consequence of nonlinear architectures due to the structure of the NTK in the infinite-width limit.
• Empirical results on ImageNet-like datasets confirm that models can be systematically biased toward non-core features when availability is increased via data augmentations.
• Availability manipulations—such as enhancing texture or background patterns—significantly increase shortcut bias, even when predictivity remains constant.
• The study identifies that features with higher singular values and greater nonlinearity in the data-generating process are more likely to be exploited by models, even when less predictive.

Figure 2: Deep nonlinear models can prefer a less-predictive but more-available feature to a more-predictive but less-available one. The color of each cell in the heatmap indicates the mean bias of a model as a function of the availability and predictivity of the shortcut feature, $z_{s}$ . The inse