[Paper Review] A Survey on Assessing the Generalization Envelope of Deep Neural Networks: Predictive Uncertainty, Out-of-distribution and Adversarial Samples

Julia Lust, Alexandru Paul Condurache|arXiv (Cornell University)|Aug 21, 2020

Adversarial Robustness in Machine Learning4 citations

TL;DR

This survey unifies three critical aspects of deep neural network (DNN) reliability—predictive uncertainty, out-of-distribution detection, and adversarial example detection—under the common framework of assessing the DNN's generalization envelope. It proposes that all three problems stem from the same root: poor generalization due to insufficient feature space separability, and advocates for a unified evaluation and methodological approach across these fields to improve trust in DNNs at inference time.

ABSTRACT

Deep Neural Networks (DNNs) achieve state-of-the-art performance on numerous applications. However, it is difficult to tell beforehand if a DNN receiving an input will deliver the correct output since their decision criteria are usually nontransparent. A DNN delivers the correct output if the input is within the area enclosed by its generalization envelope. In this case, the information contained in the input sample is processed reasonably by the network. It is of large practical importance to assess at inference time if a DNN generalizes correctly. Currently, the approaches to achieve this goal are investigated in different problem set-ups rather independently from one another, leading to three main research and literature fields: predictive uncertainty, out-of-distribution detection and adversarial example detection. This survey connects the three fields within the larger framework of investigating the generalization performance of machine learning methods and in particular DNNs. We underline the common ground, point at the most promising approaches and give a structured overview of the methods that provide at inference time means to establish if the current input is within the generalization envelope of a DNN.

Motivation & Objective

• To address the critical challenge of determining whether a deep neural network (DNN) will generalize correctly on unseen inputs at inference time.
• To unify three isolated research fields—predictive uncertainty, out-of-distribution detection, and adversarial example detection—under the shared goal of assessing the DNN’s generalization envelope.
• To identify common underlying causes of generalization failure, such as poor feature space separability and overconfidence in predictions.
• To provide a structured, comparative overview of inference-time methods that detect if an input lies within the DNN’s generalization region.
• To advocate for cross-field evaluation and methodological integration, especially highlighting metric-based approaches as most promising for multi-task generalization detection.

Proposed method

• Categorizes and maps existing inference-time methods into three domains: predictive uncertainty (near decision boundary), out-of-distribution detection (far from training data), and adversarial example detection (manipulated inputs).
• Analyzes the shared root cause: DNNs fail to generalize when feature representations do not separate inputs meaningfully, even with significant input differences.
• Proposes a unified framework where generalization detection is viewed as determining whether an input lies within the distributional and structural scope of the training data.
• Evaluates methods based on their ability to detect inputs outside the generalization envelope, emphasizing metric-based approaches that analyze feature space distances.
• Highlights limitations of confidence scores from softmax layers due to overconfidence, and advocates for alternative uncertainty estimation via feature-level analysis.
• Integrates insights from information flow analysis and distributional modeling to improve detection robustness across diverse failure modes.

Experimental results

Research questions

• RQ1 What are the common underlying causes of DNN generalization failure across predictive uncertainty, out-of-distribution, and adversarial example detection?
• RQ2 How do current inference-time detection methods differ in their ability to identify inputs outside the DNN’s generalization envelope?
• RQ3 To what extent can methods developed in one field (e.g., adversarial detection) be generalized to other fields (e.g., OOD detection) due to shared root causes?
• RQ4 Why do traditional confidence scores from DNNs (e.g., softmax probabilities) fail to reliably indicate generalization failure?
• RQ5 Which method classes—particularly metric-based or inconsistency-based—show the most promise for unified generalization envelope detection?

Key findings

• The three research fields—predictive uncertainty, out-of-distribution detection, and adversarial example detection—share a common root cause: failure to generalize due to poor feature space separability.
• Traditional confidence scores from DNNs, such as softmax probabilities, are often overconfident and thus unreliable indicators of generalization performance.
• Metric-based methods, which analyze distances in the feature space, show strong performance across multiple detection tasks, including both OOD and adversarial example detection.
• Inconsistency-based methods also show promise but are less effective than metric methods in multi-task detection scenarios.
• Despite significant progress, no single method currently achieves optimal performance across all three detection setups, and cross-field evaluation remains limited.
• The survey concludes that future research should prioritize unified evaluation frameworks and methodological integration, especially leveraging insights from adversarial example detection for broader generalization monitoring.