[Paper Review] Intra Order-preserving Functions for Calibration of Multi-Class Neural Networks

Amir Rahimi, Amirreza Shaban+3 more|arXiv (Cornell University)|Mar 15, 2020

Anomaly Detection Techniques and Applications11 references21 citations

TL;DR

This paper proposes intra order-preserving functions—learnable calibration functions that preserve the relative ranking of class scores in multi-class neural networks. By enforcing output order to match input order, the method ensures top-k predictions remain unchanged while enabling expressive, differentiable neural network architectures that outperform temperature scaling and Dirichlet calibration across diverse datasets and models.

ABSTRACT

Predicting calibrated confidence scores for multi-class deep networks is important for avoiding rare but costly mistakes. A common approach is to learn a post-hoc calibration function that transforms the output of the original network into calibrated confidence scores while maintaining the network's accuracy. However, previous post-hoc calibration techniques work only with simple calibration functions, potentially lacking sufficient representation to calibrate the complex function landscape of deep networks. In this work, we aim to learn general post-hoc calibration functions that can preserve the top-k predictions of any deep network. We call this family of functions intra order-preserving functions. We propose a new neural network architecture that represents a class of intra order-preserving functions by combining common neural network components. Additionally, we introduce order-invariant and diagonal sub-families, which can act as regularization for better generalization when the training data size is small. We show the effectiveness of the proposed method across a wide range of datasets and classifiers. Our method outperforms state-of-the-art post-hoc calibration methods, namely temperature scaling and Dirichlet calibration, in several evaluation metrics for the task.

Motivation & Objective

• To address miscalibration in deep neural networks, where high-confidence predictions often lack reliability, especially in high-stakes applications.
• To develop a flexible yet constrained family of calibration functions that preserve the top-k predictions of any pre-trained network.
• To improve generalization under small calibration dataset settings by introducing order-invariant and diagonal sub-families.
• To design a neural network architecture that can represent complex intra order-preserving functions using standard components.
• To outperform existing post-ho calibration methods like temperature scaling and Dirichlet calibration in calibration error and robustness.

Proposed method

• Introduces the concept of intra order-preserving functions, where the relative ordering of input logits is preserved in the output logits across all dimensions.
• Proposes a neural network architecture composed of standard layers (e.g., MLPs, normalization) that inherently satisfy the intra order-preserving property.
• Defines two sub-families: order-invariant functions (permutation-equivariant across input orderings) and diagonal functions (same transformation applied to each class independently).
• Uses the simplex domain to visualize and constrain function behavior, reducing hypothesis space and improving generalization.
• Employs a differentiable training objective to learn the calibration function on a held-out calibration set while preserving network accuracy.
• Applies the method post-hoc to any pre-trained deep network by replacing its final softmax layer with the learned calibration function.

Experimental results

Research questions

• RQ1 Can a general-purpose, learnable calibration function be designed that preserves the top-k predictions of any deep network?
• RQ2 How can the hypothesis space of calibration functions be constrained to improve generalization with limited calibration data?
• RQ3 What architectural and inductive bias choices (e.g., order-invariance, diagonality) lead to better calibration performance?
• RQ4 Does an intra order-preserving function architecture outperform existing post-hoc calibration methods like temperature scaling and Dirichlet calibration?
• RQ5 Can the proposed method maintain high accuracy while significantly reducing calibration error across diverse datasets and model architectures?

Key findings

• The proposed intra order-preserving (OP) method achieves the lowest marginal calibration error on average across all datasets, with a relative error of 0.213 compared to 0.725 for the next best method.
• The order-invariant (OI) sub-family achieves the best performance on 7 out of 12 datasets in marginal calibration error, including on CIFAR-100 and ImageNet.
• On CIFAR-100 with DenseNet-40, the OI method reduces marginal calibration error to 0.00000, outperforming even the uncalibrated baseline.
• The method consistently reduces expected calibration error (ECE) across all datasets, with the OI variant achieving the lowest ECE on 8 out of 12 benchmarks.
• The diagonal (Diag) variant shows strong performance on small calibration sets, indicating improved generalization through reduced parameter sharing.
• In marginal calibration error, the OI method achieves a 72.5% average relative error improvement over the uncalibrated baseline, significantly outperforming temperature scaling and Dirichlet calibration.