[Paper Review] Calibrating Deep Neural Network Classifiers on Out-of-Distribution Datasets

Zhihui Shao, Jianyi Yang+1 more|arXiv (Cornell University)|Jun 16, 2020

Adversarial Robustness in Machine Learning39 references4 citations

TL;DR

This paper proposes CCAC (Confidence Calibration with an Auxiliary Class), a post-hoc calibration method that improves deep neural network classifier confidence estimation on out-of-distribution (OOD) datasets by introducing an auxiliary class to separate misclassified samples from correct ones, significantly reducing overconfidence on OOD data. CCAC consistently outperforms prior methods in expected calibration error (ECE) and Brier score across image and text classification tasks, with a simplified variant (CCAC-S) enabling efficient transfer to new OOD datasets.

ABSTRACT

To increase the trustworthiness of deep neural network (DNN) classifiers, an accurate prediction confidence that represents the true likelihood of correctness is crucial. Towards this end, many post-hoc calibration methods have been proposed to leverage a lightweight model to map the target DNN's output layer into a calibrated confidence. Nonetheless, on an out-of-distribution (OOD) dataset in practice, the target DNN can often mis-classify samples with a high confidence, creating significant challenges for the existing calibration methods to produce an accurate confidence. In this paper, we propose a new post-hoc confidence calibration method, called CCAC (Confidence Calibration with an Auxiliary Class), for DNN classifiers on OOD datasets. The key novelty of CCAC is an auxiliary class in the calibration model which separates mis-classified samples from correctly classified ones, thus effectively mitigating the target DNN's being confidently wrong. We also propose a simplified version of CCAC to reduce free parameters and facilitate transfer to a new unseen dataset. Our experiments on different DNN models, datasets and applications show that CCAC can consistently outperform the prior post-hoc calibration methods.

Motivation & Objective

• To address the challenge of overconfident predictions in deep neural network classifiers when deployed on out-of-distribution (OOD) test data.
• To improve the reliability of prediction confidence by calibrating model outputs using post-hoc methods that account for OOD misclassification.
• To develop a calibration method that remains effective even when the test data distribution shifts significantly from the training distribution.
• To enable efficient transfer of the calibration model to new, unseen OOD datasets with minimal labeled data.
• To reduce the number of free parameters in calibration models while maintaining high performance on OOD data.

Proposed method

• CCAC introduces an auxiliary class in the calibration head to explicitly model misclassified samples, separating them from correctly classified predictions in the confidence estimation process.
• The method uses a lightweight 2-layer neural network that takes the target DNN’s logit output as input and maps it to a 19-class softmax output (18 original classes + 1 misclassification class).
• A simplified variant, CCAC-S, reduces parameters by using a single temperature scaling parameter for correct-class logits and a small neural network for the misclassification class.
• The calibration model is trained using a hybrid loss function with hyperparameters λ₁ and λ₂ to balance cross-entropy and calibration objectives on a validation set.
• Transferability is achieved by fine-tuning CCAC-S on a small number of labeled OOD samples from a new dataset, enabling adaptation without retraining the full model.
• Confidence calibration is evaluated using standard metrics: Expected Calibration Error (ECE), Brier Score (BS), AUROC, AUPR, and p.9 (confidence for correct predictions).

Experimental results

Research questions

• RQ1 Can a post-hoc calibration method effectively reduce overconfidence in deep neural network classifiers when applied to out-of-distribution test data?
• RQ2 Does introducing an auxiliary class to represent misclassified samples improve the reliability of prediction confidence on OOD data?
• RQ3 Can a simplified calibration model (CCAC-S) maintain strong performance while reducing the number of free parameters for better generalization and transferability?
• RQ4 How well can the CCAC method be transferred to a new, unseen OOD dataset with minimal labeled data?
• RQ5 Does CCAC consistently outperform existing post-hoc calibration methods in terms of ECE and Brier score on diverse OOD datasets?

Key findings

• On the 20 Newsgroups OOD dataset, CCAC reduced Expected Calibration Error (ECE) to 3.9%, significantly outperforming the next best baseline (Dirichlet, 8.6%).
• CCAC-S achieved an ECE of 3.0% on the OOD dataset, demonstrating that a simplified model with fewer parameters can still achieve state-of-the-art calibration performance.
• On the OOD dataset, CCAC improved AUROC to 0.861 and AUPR to 0.894, outperforming all baselines including temperature scaling (AUROC: 0.855, AUPR: 0.917).
• The transferred model CCAC-T achieved an ECE of 4.7% on the OOD dataset using only 320 labeled samples for fine-tuning, showing strong transferability with minimal data.
• Reliability diagrams and confidence histograms showed that CCAC produces well-calibrated confidence scores, with correct predictions concentrated at high confidence and misclassified samples clearly separated and assigned lower confidence.
• In the D1 dataset (in-distribution), CCAC achieved an ECE of 2.4% and Brier Score of 0.125, indicating strong performance even on in-distribution data, with consistent improvements over all baselines.