[Paper Review] PAC Confidence Sets for Deep Neural Networks via Calibrated Prediction

Sangdon Park, Osbert Bastani+2 more|arXiv (Cornell University)|Dec 31, 2019

Adversarial Robustness in Machine Learning30 references18 citations

TL;DR

This paper proposes a method to construct PAC (Probably Approximately Correct) confidence sets for deep neural networks by combining calibrated prediction with generalization bounds from statistical learning theory. The approach ensures that the true label is contained in the predicted confidence set with high probability, achieving theoretical guarantees while maintaining small set sizes across ImageNet, visual tracking, and reinforcement learning benchmarks.

ABSTRACT

We propose an algorithm combining calibrated prediction and generalization bounds from learning theory to construct confidence sets for deep neural networks with PAC guarantees---i.e., the confidence set for a given input contains the true label with high probability. We demonstrate how our approach can be used to construct PAC confidence sets on ResNet for ImageNet, a visual object tracking model, and a dynamics model for the half-cheetah reinforcement learning problem.

Motivation & Objective

• To address the lack of reliable confidence estimates in deep neural networks, especially in safety-critical applications.
• To develop confidence sets that are theoretically guaranteed to contain the true label with high probability (PAC guarantees).
• To combine calibrated prediction with statistical learning theory to improve both reliability and generalization of confidence sets.
• To evaluate the method on diverse benchmarks including ImageNet, visual object tracking, and reinforcement learning dynamics models.
• To demonstrate empirically valid confidence sets with low error rates and practical applicability in medical alarm systems.

Proposed method

• The method uses temperature scaling to calibrate the output probabilities of a pre-trained deep neural network.
• It applies generalization bounds from statistical learning theory to derive confidence sets that satisfy PAC guarantees.
• The confidence set $ C_T(x) $ is constructed such that $ \mathbb{P}_{(x,y)\sim D}[y \in C_T(x)] \geq 1 - \epsilon $ with probability at least $ 1 - \delta $ over validation data.
• The approach fits a single temperature parameter $ \tau $ on a validation set to improve calibration, enabling generalization guarantees.
• For classification, the confidence set is formed by selecting top-$ k $ classes with highest calibrated probabilities.
• The framework is extended to regression and reinforcement learning by adapting the confidence set construction to continuous and structured outputs.

Experimental results

Research questions

• RQ1 Can calibrated prediction be combined with generalization bounds to produce PAC confidence sets for deep neural networks?
• RQ2 How small can PAC confidence sets be made while still maintaining theoretical guarantees and empirical validity?
• RQ3 Does the proposed method improve reliability in safety-critical applications such as medical alarm systems?
• RQ4 How does the method perform across diverse tasks including image classification, visual tracking, and reinforcement learning dynamics modeling?
• RQ5 Can the method be applied to small datasets with limited validation data while maintaining low empirical error?

Key findings

• On ImageNet, the method produced confidence sets that were empirically valid, with an empirical confidence set error of $ \hat{L}(C;Z_{\text{test}}) = 0.0159 $ on the CHOP alarm dataset, below the target $ \epsilon = 0.02 $.
• For the CHOP alarm dataset, 27.94% of false alarms were suppressed using the confidence set predictor, while only 6.95% of true alarms were incorrectly suppressed.
• On the Auto MPG regression dataset, the empirical confidence set error was $ \hat{L}(C;Z_{\text{test}}) = 0.1250 $, slightly above $ \epsilon = 0.1 $, likely due to a relatively large $ \delta = 0.05 $.
• On the student grade regression dataset, the empirical error was $ \hat{L}(C;Z_{\text{test}}) = 0.0597 $, well below $ \epsilon = 0.1 $, indicating strong empirical validity.
• In visual object tracking, the confidence set predictor produced green bounding boxes that overapproximated the ground truth with high probability, demonstrating robustness.
• The method achieved small confidence set sizes on easy ImageNet images (e.g., $ |C(x)| = 1 $ for clear cases), with larger sets for more ambiguous or challenging inputs.