[Paper Review] A Unified Survey on Anomaly, Novelty, Open-Set, and Out-of-Distribution Detection: Solutions and Future Challenges

Mohammadreza Salehi, Hossein Mirzaei+4 more|arXiv (Cornell University)|Oct 26, 2021

Anomaly Detection Techniques and Applications87 citations

TL;DR

A cross-domain, comprehensive survey linking anomaly, novelty, open-set, and out-of-distribution detection, detailing common methodologies, relationships, and future research directions.

ABSTRACT

Machine learning models often encounter samples that are diverged from the training distribution. Failure to recognize an out-of-distribution (OOD) sample, and consequently assign that sample to an in-class label significantly compromises the reliability of a model. The problem has gained significant attention due to its importance for safety deploying models in open-world settings. Detecting OOD samples is challenging due to the intractability of modeling all possible unknown distributions. To date, several research domains tackle the problem of detecting unfamiliar samples, including anomaly detection, novelty detection, one-class learning, open set recognition, and out-of-distribution detection. Despite having similar and shared concepts, out-of-distribution, open-set, and anomaly detection have been investigated independently. Accordingly, these research avenues have not cross-pollinated, creating research barriers. While some surveys intend to provide an overview of these approaches, they seem to only focus on a specific domain without examining the relationship between different domains. This survey aims to provide a cross-domain and comprehensive review of numerous eminent works in respective areas while identifying their commonalities. Researchers can benefit from the overview of research advances in different fields and develop future methodology synergistically. Furthermore, to the best of our knowledge, while there are surveys in anomaly detection or one-class learning, there is no comprehensive or up-to-date survey on out-of-distribution detection, which our survey covers extensively. Finally, having a unified cross-domain perspective, we discuss and shed light on future lines of research, intending to bring these fields closer together.

Motivation & Objective

• Identify and articulate the relationships and commonalities among anomaly detection, novelty detection, open-set recognition, and out-of-distribution detection.
• Provide a cross-domain, unified methodological review with clear explanations and visuals.
• Evaluate and benchmark representative baselines to establish solid ground for current and future research.
• Discuss practical future directions focusing on fairness, adversarial robustness, privacy, data efficiency, and explainability.

Proposed method

• Present a unified categorization and cross-domain bridge to propagate ideas across fields.
• Provide mathematical and visual explanations of methods reviewed to aid understanding.
• Summarize and compare recent deep learning-based methods across AD, ND, OSR, and OOD using a common taxonomy.
• Perform comprehensive tests on existing baselines to ground current and future research directions.
• Highlight future research lines and practical necessities such as fairness, robustness, privacy, and explainability.

Experimental results

Research questions

• RQ1 What are the connections and distinctions among anomaly detection, novelty detection, open-set recognition, and out-of-distribution detection?
• RQ2 What are the dominant methodological approaches across these domains, and how can ideas be transferred between them?
• RQ3 What benchmarking practices and baselines exist, and how do they perform across cross-domain tasks?
• RQ4 What are the key future challenges and necessary research directions for reliable open-world learning?

Key findings

• The paper identifies and articulates the relationships between AD, ND, OSR, and OOD, showing they share concepts but have been studied in isolation.
• It provides a comprehensive methodological analysis with clear theoretical and visual explanations across domains.
• It conducts broad testing on existing baselines to establish solid ground for current and future research directions.
• The survey outlines plausible future lines of research, emphasizing fairness, adversarial robustness, privacy, data efficiency, and explainability.