[Paper Review] Proximity-Informed Calibration for Deep Neural Networks

Miao Xiong, Ailin Deng+5 more|arXiv (Cornell University)|Jun 7, 2023

Adversarial Robustness in Machine Learning4 citations

TL;DR

This paper introduces ProCal, a plug-and-play calibration method that mitigates proximity bias in deep neural networks by adjusting confidence scores based on sample proximity to training data neighbors. It proposes PIECE, a proximity-informed calibration metric, and demonstrates consistent improvement in calibration across diverse settings, including long-tail and distribution-shift scenarios, with 90% statistical significance.

ABSTRACT

Confidence calibration is central to providing accurate and interpretable uncertainty estimates, especially under safety-critical scenarios. However, we find that existing calibration algorithms often overlook the issue of *proximity bias*, a phenomenon where models tend to be more overconfident in low proximity data (i.e., data lying in the sparse region of the data distribution) compared to high proximity samples, and thus suffer from inconsistent miscalibration across different proximity samples. We examine the problem over 504 pretrained ImageNet models and observe that: 1) Proximity bias exists across a wide variety of model architectures and sizes; 2) Transformer-based models are relatively more susceptible to proximity bias than CNN-based models; 3) Proximity bias persists even after performing popular calibration algorithms like temperature scaling; 4) Models tend to overfit more heavily on low proximity samples than on high proximity samples. Motivated by the empirical findings, we propose ProCal, a plug-and-play algorithm with a theoretical guarantee to adjust sample confidence based on proximity. To further quantify the effectiveness of calibration algorithms in mitigating proximity bias, we introduce proximity-informed expected calibration error (PIECE) with theoretical analysis. We show that ProCal is effective in addressing proximity bias and improving calibration on balanced, long-tail, and distribution-shift settings under four metrics over various model architectures. We believe our findings on proximity bias will guide the development of *fairer and better-calibrated* models, contributing to the broader pursuit of trustworthy AI. Our code is available at: https://github.com/MiaoXiong2320/ProximityBias-Calibration.

Motivation & Objective

• To identify and quantify proximity bias—where models overconfidently misclassify low-proximity (rare, out-of-distribution) samples—across a wide range of models.
• To demonstrate that existing calibration methods like temperature scaling fail to mitigate this bias, leading to inconsistent miscalibration across proximity levels.
• To propose ProCal, a plug-and-play calibration method that jointly models confidence and proximity to improve uncertainty estimation.
• To introduce PIECE, a theoretically grounded metric that quantifies miscalibration due to proximity bias, with stronger sensitivity than standard ECE.

Proposed method

• ProCal uses a density-ratio estimation approach via kernel density estimation (KDE) on confidence-proximity pairs, distinguishing between correct and incorrect predictions.
• It computes proximity as the average L2 distance to the K nearest neighbors in the feature space (K=10–50), with robustness shown across different K values.
• For inference, ProCal recalibrates model confidence using the ratio of KDEs for correct vs. incorrect predictions, weighted by class frequency.
• The method supports both continuous and discrete input types through tailored implementations, enabling broad deployment.
• It leverages efficient approximate nearest-neighbor search (e.g., IVFFlat) for scalability on large-scale datasets.
• The calibration is applied post-hoc, making it plug-and-play and compatible with any pre-trained model.

Experimental results

Research questions

• RQ1 Does proximity bias exist across diverse model architectures and sizes, particularly in real-world safety-critical applications?
• RQ2 To what extent do standard calibration methods like temperature scaling fail to correct miscalibration in low-proximity samples?
• RQ3 Can a proximity-informed calibration metric like PIECE detect miscalibration that standard ECE misses due to cancellation effects?
• RQ4 How effective is ProCal in reducing proximity bias across balanced, long-tail, and distribution-shifted datasets?
• RQ5 What hyperparameters (e.g., K, distance measure) yield optimal performance and robustness?

Key findings

• Proximity bias is prevalent across 504 ImageNet-pretrained models, with Transformer-based models showing higher susceptibility than CNNs.
• Even after temperature scaling, models remain significantly more overconfident on low-proximity samples, indicating that standard calibration does not resolve proximity bias.
• Low-proximity samples exhibit higher overfitting rates than high-proximity samples, exacerbating miscalibration in rare or out-of-distribution cases.
• ProCal consistently improves calibration across four metrics (ECE, PIECE, accuracy, and reliability) on ImageNet, iNaturalist 2021, ImageNet-LT, MultiNLI, and ImageNet-C.
• The use of IVFFlat for proximity computation yields slightly better performance and superior efficiency, with minimal differences across distance measures like L2, cosine, or IVFPQ.
• PIECE is theoretically guaranteed to be at least as large as ECE, and equality holds only when proximity bias does not cause cancellation effects in standard ECE.