[Paper Review] Training Confidence-calibrated Classifiers for Detecting Out-of-Distribution Samples

Kimin Lee, Honglak Lee+2 more|arXiv (Cornell University)|Nov 25, 2017

Advanced Statistical Process MonitoringDecision Sciences365 citations

TL;DR

The paper proposes a confidence loss to train classifiers to produce low confidence on out-of-distribution (OOD) data and introduces a GAN that generates boundary-like OOD samples to jointly train a confident classifier and an adversarial generator, improving OOD detection without sacrificing in-distribution accuracy.

ABSTRACT

The problem of detecting whether a test sample is from in-distribution (i.e., training distribution by a classifier) or out-of-distribution sufficiently different from it arises in many real-world machine learning applications. However, the state-of-art deep neural networks are known to be highly overconfident in their predictions, i.e., do not distinguish in- and out-of-distributions. Recently, to handle this issue, several threshold-based detectors have been proposed given pre-trained neural classifiers. However, the performance of prior works highly depends on how to train the classifiers since they only focus on improving inference procedures. In this paper, we develop a novel training method for classifiers so that such inference algorithms can work better. In particular, we suggest two additional terms added to the original loss (e.g., cross entropy). The first one forces samples from out-of-distribution less confident by the classifier and the second one is for (implicitly) generating most effective training samples for the first one. In essence, our method jointly trains both classification and generative neural networks for out-of-distribution. We demonstrate its effectiveness using deep convolutional neural networks on various popular image datasets.

Motivation & Objective

• Motivate and formalize the problem of detecting out-of-distribution samples for classifiers that are often overconfident.
• Introduce a confidence loss that reduces model confidence on OOD inputs while preserving in-distribution accuracy.
• Develop an adversarial generator (GAN) to produce informative OOD samples that lie near the in-distribution boundary.
• Propose a joint training scheme where the classifier and GAN improve each other, enhancing OOD detection across datasets.
• Demonstrate improved OOD detection performance on multiple image datasets without compromising classification performance.

Proposed method

• Define a confidence loss that adds a KL divergence term forcing P_theta(y|x) to be close to uniform for out-of-distribution x, while keeping standard cross-entropy for in-distribution x.
• Introduce a GAN that (i) generates boundary-like OOD samples by minimizing a KL-based term w.r.t. the classifier, and (ii) uses a standard GAN loss to keep generated samples near the in-distribution.
• Formulate a joint objective that combines classifier loss and GAN loss, enabling alternating optimization over (G,D) and classifier parameters theta.
• Provide an alternating training algorithm (Algorithm 1) that progressively refines both the confident classifier and the adversarial generator, with convergence guarantees via monotonic objective decrease.
• Evaluate with CNNs (e.g., VGGNet, AlexNet) on CIFAR-10, SVHN, ImageNet, LSUN, and Gaussian noise, using threshold-based detectors and standard OOD metrics.

Experimental results

Research questions

• RQ1 Can a confidence-based objective improve the separability between in-distribution and out-of-distribution predictions without harming in-distribution accuracy?
• RQ2 Does training with near-boundary OOD samples (via a tailored GAN) yield better OOD detectors than using generic or synthetic OOD data?
• RQ3 Does joint training of a confident classifier and an adversarial generator produce consistent gains across multiple datasets and detectors?
• RQ4 How do different out-of-distribution choices (seen vs unseen) affect OOD detection performance when using the proposed loss?

Key findings

• The confidence loss substantially improves OOD detection metrics (e.g., TNR at 95% TPR, AUROC, AUPR) across several in-distribution/out-of-distribution pairs without harming classification accuracy.
• Realistic OOD data (e.g., LSUN, TinyImageNet) are more effective for improving detectors than synthetic noise when optimizing the KL term.
• The proposed GAN tends to generate low-density boundary samples, guiding the classifier to distinguish in- from out-of-distributions more effectively than a standard GAN.
• Joint training (classifier + GAN) outperforms using cross-entropy alone and often surpasses using a separately trained KL term on explicit OOD data.
• Guided gradient visualizations suggest that confidence-trained models focus on more informative regions of the input for near-boundary cases, aligning with improved detection.
• The method yields notable gains in TNR and AUROC across CIFAR-10 and SVHN experiments, and remains competitive on unseen out-of-distribution datasets.