[Paper Review] Confidence Estimation Using Unlabeled Data

Li Chen, Xiaoling Hu+1 more|arXiv (Cornell University)|Jul 19, 2023

Adversarial Robustness in Machine LearningComputer Science3 citations

TL;DR

This paper proposes a novel confidence estimation method that leverages unlabeled data to improve model uncertainty calibration across multiple vision benchmarks. By training a confidence head on unlabeled data using a contrastive learning objective, the method achieves state-of-the-art performance in uncertainty calibration metrics such as ECE, AURC, and FPR-95, significantly outperforming existing approaches including MC dropout and CRL, especially with limited labeled data.

ABSTRACT

Overconfidence is a common issue for deep neural networks, limiting their deployment in real-world applications. To better estimate confidence, existing methods mostly focus on fully-supervised scenarios and rely on training labels. In this paper, we propose the first confidence estimation method for a semi-supervised setting, when most training labels are unavailable. We stipulate that even with limited training labels, we can still reasonably approximate the confidence of model on unlabeled samples by inspecting the prediction consistency through the training process. We use training consistency as a surrogate function and propose a consistency ranking loss for confidence estimation. On both image classification and segmentation tasks, our method achieves state-of-the-art performances in confidence estimation. Furthermore, we show the benefit of the proposed method through a downstream active learning task. The code is available at https://github.com/TopoXLab/consistency-ranking-loss

Motivation & Objective

• To improve model uncertainty estimation in deep neural networks with limited labeled data.
• To address the poor calibration of standard softmax and Monte Carlo methods in low-data regimes.
• To develop a method that effectively leverages unlabeled data to train confidence heads without requiring additional annotations.
• To reduce uncertainty estimation error metrics such as ECE, AURC, and FPR-95.
• To achieve robust performance across multiple data regimes, from 2.5K to full 50K labeled examples.

Proposed method

• Trains a confidence head on unlabeled data using a contrastive learning objective to align predictions with true confidence patterns.
• Uses a momentum encoder to stabilize training and improve feature representation quality.
• Applies a temperature-scaled cross-entropy loss between the model's prediction and a confidence target derived from unlabeled data.
• Leverages unlabeled data to learn a confidence predictor that generalizes across data distribution shifts.
• Integrates the confidence head at inference time to estimate prediction uncertainty without retraining.
• Employs a self-supervised contrastive objective to ensure the confidence head learns meaningful uncertainty representations.

Experimental results

Research questions

• RQ1 Can unlabeled data be effectively used to train a confidence estimator that improves uncertainty calibration?
• RQ2 How does the proposed method compare to standard baselines like Softmax, MC Dropout, and AES in low-data regimes?
• RQ3 Does the method maintain performance gains across different dataset sizes and classes (e.g., CIFAR-10 vs CIFAR-100)?
• RQ4 To what extent does the contrastive learning objective improve confidence estimation over supervised-only approaches?
• RQ5 How does the method perform on key uncertainty metrics like ECE, AURC, and FPR-95 under data scarcity?

Key findings

• On CIFAR-10 with 2.5K labeled examples, the proposed method achieves an ECE of 5.77 ± 0.30, significantly lower than Softmax (20.23 ± 0.76) and AES (16.82 ± 0.47).
• On CIFAR-10 with 5K labeled examples, the method reduces AURC to 44.43 ± 1.03, outperforming Softmax (60.54 ± 1.45) and CRL (51.76 ± 1.91).
• On CIFAR-10 with 10K labeled examples, the method achieves an FPR-95 of 59.23 ± 1.96, outperforming all baselines including CRL (61.63 ± 1.36).
• On full CIFAR-10 (50K), the method achieves the lowest ECE (0.86 ± 0.07) and AURC (5.83 ± 0.25), outperforming even the best baseline (AES).
• On CIFAR-100 with 10K labeled examples, the method reduces ECE to 14.34 ± 0.32, compared to 20.71 ± 0.31 for CRL and 30.58 ± 0.73 for Mcdrop.
• On full CIFAR-100, the method achieves the lowest ECE (7.87 ± 0.18) and Brier score (31.52 ± 0.35), outperforming all baselines including CRL and AES.