[Paper Review] Sensitivity and Generalization in Neural Networks: an Empirical Study

Roman Novak, Yasaman Bahri+3 more|arXiv (Cornell University)|Feb 23, 2018

Neural Networks and Applications43 references224 citations

TL;DR

The paper empirically links neural network generalization to input-output Jacobian sensitivity and linear-region transitions, showing robustness near the data manifold correlates with better generalization across many architectures and settings.

ABSTRACT

In practice it is often found that large over-parameterized neural networks generalize better than their smaller counterparts, an observation that appears to conflict with classical notions of function complexity, which typically favor smaller models. In this work, we investigate this tension between complexity and generalization through an extensive empirical exploration of two natural metrics of complexity related to sensitivity to input perturbations. Our experiments survey thousands of models with various fully-connected architectures, optimizers, and other hyper-parameters, as well as four different image classification datasets. We find that trained neural networks are more robust to input perturbations in the vicinity of the training data manifold, as measured by the norm of the input-output Jacobian of the network, and that it correlates well with generalization. We further establish that factors associated with poor generalization $-$ such as full-batch training or using random labels $-$ correspond to lower robustness, while factors associated with good generalization $-$ such as data augmentation and ReLU non-linearities $-$ give rise to more robust functions. Finally, we demonstrate how the input-output Jacobian norm can be predictive of generalization at the level of individual test points.

Motivation & Objective

• Investigate the tension between model capacity and generalization in over-parameterized networks.
• Define and evaluate sensitivity metrics related to input perturbations.
• Examine the relationship between sensitivity and generalization across various architectures, optimizers, and hyper-parameters.
• Assess whether sensitivity metrics can predict generalization at the level of individual test points.

Proposed method

• Define two sensitivity metrics for fully-connected networks: Jacobian norm of the softmax outputs and the number of linear-region transitions along input trajectories.
• Compute average Jacobian Frobenius norm around test points to measure local sensitivity.
• Count transitions between linear regions by encoding neuron activation patterns along trajectories close to the data manifold.
• Compare sensitivity on and off the data manifold using circular trajectories and ellipses through training points.
• Analyze sensitivity under different generalization-affecting factors (e.g., data augmentation, label quality, ReLU vs saturating activations, mini-batch vs full-batch training).
• Perform large-scale experiments across thousands of fully-connected models on multiple image classification datasets.

Experimental results

Research questions

• RQ1 Does neural network generalization correlate with input-output Jacobian sensitivity?
• RQ2 How do factors that influence generalization (e.g., data augmentation, labeling, activation functions, batch size) affect sensitivity?
• RQ3 Is sensitivity predictive of generalization at the level of individual test points?
• RQ4 How do sensitivity measures compare when evaluating models that differ in architecture and optimization hyper-parameters?

Key findings

• Jacobian norm correlates with generalization across diverse settings and datasets.
• Sensitivity is higher off the data manifold and reduces near training data points, indicating more robust functions in that region.
• Factors improving generalization (correct labels, data augmentation, ReLU activations, mini-batch optimization) are consistently accompanied by reduced sensitivity.
• Transition density alone is not sufficient to compare networks of different sizes; architecture size influences the count of transitions.
• Jacobian norm at individual test points correlates with cross-entropy loss, suggesting per-point predictive utility for active learning and confidence estimation.
• The study provides extensive empirical evidence linking local geometry of learned functions to generalization in image classification.