[论文解读] Generalized Out-of-Distribution Detection: A Survey

Jingkang Yang, Kaiyang Zhou+2 more|arXiv (Cornell University)|Oct 21, 2021

Anomaly Detection Techniques and Applications参考文献 177被引用 323

一句话总结

对 OOD 检测的综合综述，澄清 OSR 与开放世界识别的区别，整合基础模型、基准测试实践和理论基础，提供实用指南以及 OpenOOD 资源。

ABSTRACT

Out-of-distribution (OOD) detection is critical to ensuring the reliability and safety of machine learning systems. For instance, in autonomous driving, we would like the driving system to issue an alert and hand over the control to humans when it detects unusual scenes or objects that it has never seen during training time and cannot make a safe decision. The term, OOD detection, first emerged in 2017 and since then has received increasing attention from the research community, leading to a plethora of methods developed, ranging from classification-based to density-based to distance-based ones. Meanwhile, several other problems, including anomaly detection (AD), novelty detection (ND), open set recognition (OSR), and outlier detection (OD), are closely related to OOD detection in terms of motivation and methodology. Despite common goals, these topics develop in isolation, and their subtle differences in definition and problem setting often confuse readers and practitioners. In this survey, we first present a unified framework called generalized OOD detection, which encompasses the five aforementioned problems, i.e., AD, ND, OSR, OOD detection, and OD. Under our framework, these five problems can be seen as special cases or sub-tasks, and are easier to distinguish. We then review each of these five areas by summarizing their recent technical developments, with a special focus on OOD detection methodologies. We conclude this survey with open challenges and potential research directions.

研究动机与目标

• 澄清 OSR、开放世界识别及相关范式之间的关系。
• 勾勒一个统一的 OOD 检测框架，考虑在 X（输入）和 Y（标签）上的分布漂移。
• 综述关键方法、基准和数据集；讨论 OpenOOD 及实际实验指南。
• 讨论基础模型和保形预测在 OOD 检测中的作用；指出未来方向。

提出的方法

• 给出分布 P(X) 与 P(Y) 及其漂移的形式化定义（协变量漂移 vs 语义漂移）。
• 在一个通用框架中对 OOD 任务进行分类，并与相关设定（OSR、AD/ND）进行对比。
• 总结最近的方法，包括基于基础模型的方法和零样本/检测技术。
• 讨论基准测试实践和真实世界数据集；推动将 OpenOOD 作为统一的基准测试平台。
• 融入理论视角，讨论保形预测作为提高 OOD 置信度的潜在工具。

实验结果

研究问题

• RQ1 如何在一个统一框架下将 OSR、开放世界识别与 OOD 检测统一起来？
• RQ2 协变量漂移与语义漂移对 OOD 检测器性能的影响是什么，以及我们能否推广到全谱 OOD？
• RQ3 基础模型和视觉-语言模型如何影响 OOD 检测，以及哪些微调策略有助于或妨碍性能？
• RQ4 哪些基准和数据集最能反映现实世界的 OOD 挑战，以及 benchmarking 应如何进行？
• RQ5 哪些理论基础和概率保证适用于现代模型中的 OOD 检测？

主要发现

• 当前检测器往往对协变量漂移比对语义漂移更敏感，这促使考虑全谱 OOD。
• 开放世界与基础模型的讨论日益相关；OpenOOD 提供了统一的基准测试框架并提供报告。
• 保形预测被确定为在 OOD 情境下提供置信度的有前景的工具。
• 基准测试应强调现实数据集（如 WILDS、ImageNet 基准的修正）及真实世界场景。
• 本综述强调需要将 OOD 检测与大型多模态模型和世界模型相连接以提升可靠性。