[Paper Review] Distinguishing rule- and exemplar-based generalization in learning systems

Ishita Dasgupta, Erin Grant+1 more|arXiv (Cornell University)|Oct 8, 2021

Neural Networks and Applications4 citations

TL;DR

This paper introduces a cognitive psychology-inspired experimental protocol to distinguish between rule-based and exemplar-based generalization in machine learning systems. By isolating two inductive biases—feature-level bias (preference for certain features) and exemplar vs. rule bias (use of feature-sparse vs. dense representations)—the authors show that standard neural networks are both feature-biased and exemplar-based, highlighting limitations in systematic generalization and fairness.

ABSTRACT

Machine learning systems often do not share the same inductive biases as humans and, as a result, extrapolate or generalize in ways that are inconsistent with our expectations. The trade-off between exemplar- and rule-based generalization has been studied extensively in cognitive psychology; in this work, we present a protocol inspired by these experimental approaches to probe the inductive biases that control this tradeoff in category-learning systems. We isolate two such inductive biases: feature-level bias (differences in which features are more readily learned) and exemplar or rule bias (differences in how these learned features are used for generalization). We find that standard neural network models are feature-biased and exemplar-based, and discuss the implications of these findings for machine learning research on systematic generalization, fairness, and data augmentation.

Motivation & Objective

• To identify and isolate two key inductive biases—feature-level bias and exemplar vs. rule bias—that govern how machine learning systems generalize beyond training data.
• To develop an experimental protocol inspired by cognitive psychology to probe these biases in a controlled, diagnostic manner.
• To evaluate how standard machine learning models (e.g., neural networks, Gaussian processes) exhibit these biases across synthetic, image, and text domains.
• To demonstrate that existing metrics like worst-group accuracy and spurious correlation are insufficient for capturing these underlying inductive biases.
• To explore the normative implications of these biases for systematic generalization, fairness, and data augmentation in machine learning.

Proposed method

• Design a synthetic data generation protocol where a discriminant feature (e.g., shape) determines category labels, while a distractor feature (e.g., color) is causally irrelevant but co-occurring in training data.
• Train and evaluate diverse model families: feedforward neural networks (NNs), linear generalized linear models (GLMs), and Gaussian processes (GPs), with controlled regularization and kernel lengthscale parameters.
• Use a novel metric, exemplar vs. rule propensity (EvR), to quantify whether a model generalizes based on similarity to training exemplars (exemplar-based) or on abstract rules (rule-based).
• Control for feature-level bias by ensuring equal feature importance in synthetic data, allowing isolation of exemplar vs. rule bias.
• Apply the protocol to real-world domains (CelebA and IMDb) to assess generalization behavior in more complex, naturalistic settings.
• Validate the protocol using explicitly rule-based (linear models with L1 regularization) and exemplar-based (GPs with short lengthscale) models as benchmarks.

Experimental results

Research questions

• RQ1 How can we isolate and measure two distinct inductive biases—feature-level bias and exemplar vs. rule bias—in machine learning systems?
• RQ2 To what extent do standard neural networks exhibit exemplar-based generalization and feature-level bias in category learning tasks?
• RQ3 How do model hyperparameters such as L1/L2 regularization and kernel lengthscale influence the exemplar vs. rule propensity of a model?
• RQ4 Are existing fairness and generalization metrics (e.g., worst-group accuracy, spurious correlation) sufficient to capture the underlying inductive biases of models?
• RQ5 What are the implications of these biases for systematic generalization, fairness, and data augmentation in real-world machine learning applications?

Key findings

• Standard neural networks exhibit a strong exemplar-based bias, generalizing by similarity to training examples rather than by abstract rules.
• Neural networks are also feature-biased, showing a preference for certain features (e.g., shape over color) even when both are equally predictive.
• L1 regularization in generalized linear models reduces exemplar vs. rule propensity to zero, confirming that feature sparsity induces rule-based behavior.
• Gaussian processes with shorter lengthscale parameters show higher exemplar propensity, confirming that local decision boundaries promote exemplar-based generalization.
• The proposed EvR metric successfully captures the trade-off between exemplar and rule-based generalization across model families and data domains.
• Existing metrics like worst-group accuracy and spurious correlation fail to diagnose the underlying inductive biases, indicating a need for more nuanced evaluation protocols.