[Paper Review] Addressing Failure Prediction by Learning Model Confidence

Charles Corbière, Nicolas Thome+3 more|arXiv (Cornell University)|Oct 1, 2019

Adversarial Robustness in Machine Learning42 references107 citations

TL;DR

The paper introduces True Class Probability (TCP) as a confidence criterion for failure prediction in classification and segmentation, and presents ConfidNet, a learned confidence predictor that outperforms MCP and Bayesian/ensemble baselines across multiple datasets.

ABSTRACT

Assessing reliably the confidence of a deep neural network and predicting its failures is of primary importance for the practical deployment of these models. In this paper, we propose a new target criterion for model confidence, corresponding to the True Class Probability (TCP). We show how using the TCP is more suited than relying on the classic Maximum Class Probability (MCP). We provide in addition theoretical guarantees for TCP in the context of failure prediction. Since the true class is by essence unknown at test time, we propose to learn TCP criterion on the training set, introducing a specific learning scheme adapted to this context. Extensive experiments are conducted for validating the relevance of the proposed approach. We study various network architectures, small and large scale datasets for image classification and semantic segmentation. We show that our approach consistently outperforms several strong methods, from MCP to Bayesian uncertainty, as well as recent approaches specifically designed for failure prediction.

Motivation & Objective

• Motivate reliable failure prediction for deep networks in practical applications.
• Propose True Class Probability (TCP) as a theoretically grounded confidence criterion.
• Learn TCP-based confidence scores with a dedicated neural network (ConfidNet) built on top of a fixed classifier.
• Demonstrate empirical gains of TCP/ConfidNet over MCP, MCDropout, and Trust Score across diverse datasets and architectures.

Proposed method

• Define TCP as the probability of the true class given input and model weights, and derive theoretical guarantees for correct/incorrect predictions.
• Introduce ConfidNet, a confidence network that regresses toward c*(x,y*) = P(Y=y*|w,x) using L2 loss, built on top of a fixed classifier.
• Optionally use a normalized variant TCP^r = P(Y=y*|w,x)/P(Y=ŷ|w,x) and compare learning dynamics.
• Adopt a two-stage training scheme: train ConfidNet with the classifier frozen, then optionally fine-tune the shared encoder with a decoupled feature extractor for confidence estimation.
• Evaluate with various architectures and datasets (MNIST, SVHN, CIFAR-10/100, CamVid) using AUPR-Error, AUPR-Success, FPR@TPR95, and AUROC.

Experimental results

Research questions

• RQ1 Can TCP provide more reliable failure-related signaling than MCP for deep classifiers?
• RQ2 Does learning a TCP-based confidence predictor (ConfidNet) improve failure detection across image classification and semantic segmentation tasks?
• RQ3 How do TCP and its normalized variant TCP^r compare to MCP, Bayesian methods, and distance-based scores (Trust Score) in terms of ranking and calibration for failures?
• RQ4 What is the impact of learning scheme choices (training ConvNet encoder, loss type, and data allocation) on ConfidNet performance?
• RQ5 Are TCP-based methods robust to dataset size and model complexity across a range of tasks?

Key findings

• TCP-based ranking yields better separability between correct and incorrect predictions than MCP, improving failure detection.
• ConfidNet consistently outperforms MCP, MCDropout, and Trust Score across MNIST, SVHN, CIFAR-10/100, and CamVid in AUPR-Error and related metrics.
• Fine-tuning the ConvNet encoder after training ConfidNet yields additional gains (~1–2 percentage points) over the base ConfidNet.
• Training ConfidNet on train data generally outperforms training on validation data for most datasets, with some exceptions linked to dataset characteristics.
• Learning with TCP (vs BCE or TCP^r) provides robust improvements, especially on larger datasets; TCP^r offers benefits on some large-scale segmentation tasks.
• ConfidNet offers favorable selective classification behavior (risk-coverage) at higher coverage levels, enabling better identification of critical failures.