[Paper Review] On the Reproducibility of Neural Network Predictions

Srinadh Bhojanapalli, Michael J. Wilber+5 more|arXiv (Cornell University)|Feb 5, 2021

Neural Networks and ApplicationsComputer Science34 references17 citations

TL;DR

This paper addresses prediction churn in neural networks caused by training randomness, proposing a dual method: minimum entropy regularization to boost prediction confidence and a novel symmetric KL-based co-distillation to enhance model agreement. The approach reduces churn by up to 50% on ImageNet while improving accuracy and calibration, outperforming standard distillation and ensembling at lower training cost.

ABSTRACT

Standard training techniques for neural networks involve multiple sources of randomness, e.g., initialization, mini-batch ordering and in some cases data augmentation. Given that neural networks are heavily over-parameterized in practice, such randomness can cause {\em churn} -- for the same input, disagreements between predictions of the two models independently trained by the same algorithm, contributing to the `reproducibility challenges' in modern machine learning. In this paper, we study this problem of churn, identify factors that cause it, and propose two simple means of mitigating it. We first demonstrate that churn is indeed an issue, even for standard image classification tasks (CIFAR and ImageNet), and study the role of the different sources of training randomness that cause churn. By analyzing the relationship between churn and prediction confidences, we pursue an approach with two components for churn reduction. First, we propose using \emph{minimum entropy regularizers} to increase prediction confidences. Second, \changes{we present a novel variant of co-distillation approach~\citep{anil2018large} to increase model agreement and reduce churn}. We present empirical results showing the effectiveness of both techniques in reducing churn while improving the accuracy of the underlying model.

Motivation & Objective

• To investigate the root causes of prediction churn in deep neural networks despite identical architectures and data.
• To quantify how training randomness—such as weight initialization and mini-batch ordering—affects model disagreement on test examples.
• To develop methods that reduce churn without eliminating randomness, ensuring stable predictions across training runs.
• To improve model confidence and agreement through regularization and distillation, enhancing reproducibility in real-world ML applications.
• To evaluate the trade-offs between churn reduction, calibration, and accuracy, especially in comparison to existing distillation and ensembling techniques.

Proposed method

• Introduces minimum entropy regularization to increase prediction confidence by minimizing the entropy of model outputs during training.
• Proposes a new co-distillation framework using symmetric Kullback-Leibler (SKL) divergence as the loss to align predictions between two student models.
• Combines entropy regularization and SKL-based co-distillation into a single training objective to jointly improve confidence and agreement.
• Employs a two-branch training setup where two models are trained simultaneously with shared data and knowledge distillation loss.
• Uses a weighted combination of cross-entropy loss, entropy regularization, and symmetric KL loss to balance accuracy, confidence, and agreement.
• Applies the method across multiple benchmarks: CIFAR-10, CIFAR-100, SVHN, iNaturalist, and ImageNet, with ablation studies on hyperparameters and weight decay.

Experimental results

Research questions

• RQ1 What are the primary sources of prediction churn in neural networks trained with standard stochastic optimization?
• RQ2 How does model prediction confidence correlate with churn, and can improving confidence reduce disagreement between independently trained models?
• RQ3 Can co-distillation with symmetric KL divergence reduce prediction variance and model disagreement more effectively than standard cross-entropy-based distillation?
• RQ4 Does combining entropy regularization and symmetric KL co-distillation lead to better churn reduction and improved accuracy compared to individual components or existing methods?
• RQ5 How do the proposed methods affect model calibration, especially in comparison to label smoothing and ensemble distillation?

Key findings

• Prediction churn remains significant even with fixed data and architecture, with up to 20% disagreement across five independently trained ResNet-v2-50 models on ImageNet.
• The proposed symmetric KL-based co-distillation (Co-distill_SKL) reduced churn by 50% on ImageNet compared to baseline, outperforming the standard Co-distill_CE method.
• Minimum entropy regularization reduced churn by improving prediction confidence, though it increased calibration error (ECE from 3.71 to 6.17 on ImageNet).
• The combined method of entropy regularization and Co-distill_SKL achieved the lowest churn and competitive accuracy, with ECE of 3.89 on ImageNet—close to the 2.14 of 2-ensemble distillation but at half the training cost.
• Co-distill_SKL reduced ECE from 11.97 (Co-distill_CE) to 2.44 on ImageNet, showing improved calibration despite lower confidence.
• The ablation study confirmed the complementary nature of the two components: entropy regularization reduced entropy, while co-distillation reduced prediction variance, leading to superior performance when combined.