[Paper Review] Limitations of Neural Collapse for Understanding Generalization in Deep Learning

Like Hui, Mikhail Belkin+1 more|arXiv (Cornell University)|Feb 17, 2022

Neural Networks and Applications10 citations

TL;DR

This paper investigates the limitations of Neural Collapse in explaining generalization in deep learning. It refines Neural Collapse into train-set and test-set variants, demonstrating that while collapse reliably occurs during training (optimization), it fails to emerge on the test distribution. Crucially, increased collapse correlates with worse downstream transfer performance, challenging the assumption that collapse enhances representation learning. The authors further observe a 'cascading collapse' in deeper layers, suggesting collapse may propagate from last to earlier layers.

ABSTRACT

The recent work of Papyan, Han, & Donoho (2020) presented an intriguing "Neural Collapse" phenomenon, showing a structural property of interpolating classifiers in the late stage of training. This opened a rich area of exploration studying this phenomenon. Our motivation is to study the upper limits of this research program: How far will understanding Neural Collapse take us in understanding deep learning? First, we investigate its role in generalization. We refine the Neural Collapse conjecture into two separate conjectures: collapse on the train set (an optimization property) and collapse on the test distribution (a generalization property). We find that while Neural Collapse often occurs on the train set, it does not occur on the test set. We thus conclude that Neural Collapse is primarily an optimization phenomenon, with as-yet-unclear connections to generalization. Second, we investigate the role of Neural Collapse in feature learning. We show simple, realistic experiments where training longer leads to worse last-layer features, as measured by transfer-performance on a downstream task. This suggests that neural collapse is not always desirable for representation learning, as previously claimed. Finally, we give preliminary evidence of a "cascading collapse" phenomenon, wherein some form of Neural Collapse occurs not only for the last layer, but in earlier layers as well. We hope our work encourages the community to continue the rich line of Neural Collapse research, while also considering its inherent limitations.

Motivation & Objective

• To clarify the ambiguity in Neural Collapse research by distinguishing between train-set and test-set collapse.
• To investigate whether Neural Collapse genuinely explains generalization or is merely an optimization artifact.
• To evaluate the impact of Neural Collapse on representation learning using transfer learning benchmarks.
• To explore whether collapse extends beyond the last layer to earlier network layers.
• To provide a critical assessment of the Neural Collapse research program, highlighting its strengths and limitations.

Proposed method

• Propose refined, precise formulations of Neural Collapse: weak and strong versions for both train and test sets.
• Conduct empirical evaluation of feature collapse on training and test sets across multiple architectures (e.g., ResNet18) and datasets (e.g., CIFAR-10).
• Measure the correlation between the degree of train-set collapse and downstream transfer performance on linear probes and fine-tuning tasks.
• Perform controlled experiments on fully connected networks to observe the temporal sequence of collapse across hidden layers.
• Use variance-based metrics to quantify collapse, measuring the concentration of features per class.
• Apply standard deep learning training protocols (SGD with momentum, cosine learning rate decay) to ensure realistic training dynamics.

Experimental results

Research questions

• RQ1 Does Neural Collapse occur on the test distribution, or is it confined to the training set?
• RQ2 To what extent does Neural Collapse on the training set predict or improve generalization performance?
• RQ3 Is increased Neural Collapse beneficial for representation learning, as measured by downstream transfer accuracy?
• RQ4 Can Neural Collapse propagate from the last layer to earlier layers in deep networks?
• RQ5 What are the theoretical and empirical limits of extending the Neural Collapse conjecture to generalization?

Key findings

• Neural Collapse reliably occurs on the training set across multiple architectures and datasets, confirming the original observation.
• Test-set collapse does not occur in practice, even when train-set collapse is strong, indicating that collapse is not a generalization property.
• Increased train-set collapse is anti-correlated with downstream transfer performance, suggesting that more collapse can degrade representation quality.
• Empirical results show that models with higher feature variance (less collapse) achieve better linear probe and fine-tuning accuracy on downstream tasks.
• Preliminary evidence suggests a 'cascading collapse' phenomenon, where collapse emerges sequentially from the last layer backward through earlier layers during training.
• Strong test-set collapse is theoretically infeasible under standard assumptions, while weak test collapse is possible but not observed in practice.