[Paper Review] Generalized Out-of-Distribution Detection: A Survey

Jingkang Yang, Kaiyang Zhou+2 more|arXiv (Cornell University)|Oct 21, 2021

Anomaly Detection Techniques and Applications177 references323 citations

TL;DR

A comprehensive survey of OOD detection, clarifying OSR vs Open-World Recognition, integrating foundation models, benchmarking practices, and theoretical foundations, with practical guidance and OpenOOD resources.

ABSTRACT

Out-of-distribution (OOD) detection is critical to ensuring the reliability and safety of machine learning systems. For instance, in autonomous driving, we would like the driving system to issue an alert and hand over the control to humans when it detects unusual scenes or objects that it has never seen during training time and cannot make a safe decision. The term, OOD detection, first emerged in 2017 and since then has received increasing attention from the research community, leading to a plethora of methods developed, ranging from classification-based to density-based to distance-based ones. Meanwhile, several other problems, including anomaly detection (AD), novelty detection (ND), open set recognition (OSR), and outlier detection (OD), are closely related to OOD detection in terms of motivation and methodology. Despite common goals, these topics develop in isolation, and their subtle differences in definition and problem setting often confuse readers and practitioners. In this survey, we first present a unified framework called generalized OOD detection, which encompasses the five aforementioned problems, i.e., AD, ND, OSR, OOD detection, and OD. Under our framework, these five problems can be seen as special cases or sub-tasks, and are easier to distinguish. We then review each of these five areas by summarizing their recent technical developments, with a special focus on OOD detection methodologies. We conclude this survey with open challenges and potential research directions.

Motivation & Objective

• Clarify the relationships among OSR, open-world recognition, and related paradigms.
• Outline a unified framework for OOD detection with distribution shifts in X (inputs) and Y (labels).
• Survey key methods, benchmarks, and datasets; discuss OpenOOD and practical experimental guidance.
• Discuss the role of foundation models and conformal prediction in OOD detection; identify future directions.

Proposed method

• Provide formal definitions of distributions P(X) and P(Y) and their shifts (covariate vs semantic).
• Classify OOD tasks within a general framework and contrast with related settings (OSR, AD/ND).
• Summarize recent methods, including foundation-model-based approaches and zero-shot/detection techniques.
• Discuss benchmarking practices and real-world datasets; promote OpenOOD as a unified benchmarking platform.
• Incorporate theoretical perspectives and discuss Conformal Prediction as a potential tool for OOD confidence.

Experimental results

Research questions

• RQ1 How can OSR, open-world recognition, and OOD detection be unified under a single framework?
• RQ2 What is the impact of covariate versus semantic shifts on OOD detector performance, and can we generalize to full-spectrum OOD?
• RQ3 How do foundation and vision-language models influence OOD detection, and what tuning strategies help or hinder performance?
• RQ4 What benchmarks and datasets best reflect real-world OOD challenges, and how should benchmarking be conducted?
• RQ5 What theoretical foundations and probabilistic guarantees apply to OOD detection in modern models?

Key findings

• Current detectors often show higher sensitivity to covariate shifts than semantic shifts, motivating full-spectrum OOD considerations.
• Open-world and foundation-model discussions are increasingly relevant; OpenOOD provides a unified benchmarking framework and reports.
• Conformal prediction is identified as a promising tool for providing confidence measures in OOD contexts.
• Benchmarking should emphasize realistic datasets (e.g., WILDS, ImageNet benchmarks corrections) and real-world scenarios.
• The survey emphasizes the need to connect OOD detection with large multi-modal models and world models for reliability.