[Paper Review] Model Similarity Mitigates Test Set Overuse

Horia Mania, J. J. Miller+3 more|arXiv (Cornell University)|May 29, 2019

Machine Learning and Data Classification17 references4 citations

TL;DR

This paper proposes that model similarity—where models make highly correlated predictions—mitigates overfitting to test sets, enabling far more reuse than traditional generalization bounds allow. By introducing a similarity-aware generalization bound, the authors show that high model similarity can increase the number of safe test set reuses by up to four orders of magnitude, explaining why benchmarks like ImageNet remain valid despite extensive reuse.

ABSTRACT

Excessive reuse of test data has become commonplace in today's machine learning workflows. Popular benchmarks, competitions, industrial scale tuning, among other applications, all involve test data reuse beyond guidance by statistical confidence bounds. Nonetheless, recent replication studies give evidence that popular benchmarks continue to support progress despite years of extensive reuse. We proffer a new explanation for the apparent longevity of test data: Many proposed models are similar in their predictions and we prove that this similarity mitigates overfitting. Specifically, we show empirically that models proposed for the ImageNet ILSVRC benchmark agree in their predictions well beyond what we can conclude from their accuracy levels alone. Likewise, models created by large scale hyperparameter search enjoy high levels of similarity. Motivated by these empirical observations, we give a non-asymptotic generalization bound that takes similarity into account, leading to meaningful confidence bounds in practical settings.

Motivation & Objective

• To explain why test sets like ImageNet remain valid despite extensive reuse, contrary to theoretical expectations of overfitting.
• To investigate the empirical extent of model similarity across popular benchmarks and hyperparameter searches.
• To develop a new generalization bound that incorporates model similarity to provide tighter, more practical confidence intervals.
• To quantify how much model similarity improves the number of safe test set reuses in adaptive and non-adaptive settings.
• To explore whether structural dependencies between models can further reduce overfitting risk.

Proposed method

• Empirically measure pairwise prediction similarity across 1,000+ ImageNet and CIFAR-10 models, including those from random hyperparameter search and neural architecture search.
• Define a similarity measure based on agreement in predictions across test samples, independent of accuracy.
• Introduce a new non-asymptotic generalization bound that accounts for model similarity via a covering number of the model family.
• Derive a refined union bound that replaces the standard √(log k / n) term with a similarity-dependent term, reducing overfitting risk.
• Apply the bound to estimate the maximum number of models that can be safely evaluated on a test set with a given error tolerance.
• Compare the similarity-based bound to the standard union bound and naive Bayes approximation, showing significant improvements in practical settings.

Experimental results

Research questions

• RQ1 To what extent do models trained on ImageNet and CIFAR-10 exhibit prediction similarity beyond what their accuracy suggests?
• RQ2 How does model similarity affect the generalization error bound in adaptive test set reuse scenarios?
• RQ3 Can a similarity-aware generalization bound significantly increase the number of safe test set reuses compared to standard bounds?
• RQ4 Does high model similarity persist during large-scale hyperparameter and neural architecture search, even when models are selected adaptively?
• RQ5 What is the impact of additional structural assumptions on model predictions in further improving the similarity-based bound?

Key findings

• ImageNet and CIFAR-10 models exhibit significantly higher prediction agreement than expected from their accuracy alone, with mean pairwise similarity reaching 97.5% in some cases.
• In random hyperparameter search on CIFAR-10, model checkpoints remain highly similar, with similarity levels far exceeding random chance.
• Neural architecture search models show mean similarity of 97.6% among top-performing configurations, indicating strong structural convergence.
• The proposed similarity-based generalization bound increases the number of safe test set reuses by up to 12× compared to the standard union bound.
• Under additional modeling assumptions, the bound can improve the number of safe reuses by up to 34,000× compared to the baseline.
• Even in high-accuracy regimes, model similarity allows for significantly more test set evaluations without overfitting, explaining the longevity of benchmarks like ImageNet.