[Paper Review] Out-of-Distribution Detection in Multi-Label Datasets using Latent Space of $\beta$-VAE

Vijaya Kumar Sundar, Shreyas Ramakrishna+3 more|arXiv (Cornell University)|Mar 10, 2020

Anomaly Detection Techniques and Applications13 references4 citations

TL;DR

This paper proposes a novel method for out-of-distribution (OOD) detection in multi-label autonomous driving datasets using the disentangled latent space of β-VAE. By training separate β-VAEs on data partitions defined by specific generative factors (e.g., time-of-day, traffic, pedestrians), and selecting latent variables sensitive to those factors, the approach uses KL-divergence from a learned threshold to detect distributional shifts with high accuracy, achieving up to 100% OOD detection on pedestrian and traffic partitions in the nuScenes dataset.

ABSTRACT

Learning Enabled Components (LECs) are widely being used in a variety of perception based autonomy tasks like image segmentation, object detection, end-to-end driving, etc. These components are trained with large image datasets with multimodal factors like weather conditions, time-of-day, traffic-density, etc. The LECs learn from these factors during training, and while testing if there is variation in any of these factors, the components get confused resulting in low confidence predictions. The images with factors not seen during training is commonly referred to as Out-of-Distribution (OOD). For safe autonomy it is important to identify the OOD images, so that a suitable mitigation strategy can be performed. Classical one-class classifiers like SVM and SVDD are used to perform OOD detection. However, the multiple labels attached to the images in these datasets, restricts the direct application of these techniques. We address this problem using the latent space of the $\beta$-Variational Autoencoder ($\beta$-VAE). We use the fact that compact latent space generated by an appropriately selected $\beta$-VAE will encode the information about these factors in a few latent variables, and that can be used for computationally inexpensive detection. We evaluate our approach on the nuScenes dataset, and our results shows the latent space of $\beta$-VAE is sensitive to encode changes in the values of the generative factor.

Motivation & Objective

• To address the challenge of detecting out-of-distribution (OOD) images in multi-label autonomous driving datasets where traditional one-class OOD methods fail due to label overlap.
• To enable reliable OOD detection in real-world multimodal datasets like nuScenes, which contain complex, overlapping factors such as weather, time-of-day, and traffic density.
• To develop a scalable, computationally efficient method that leverages disentangled representations from β-VAE to detect distributional shifts in specific generative factors.

Proposed method

• Partition the multi-label dataset (e.g., nuScenes) into subsets based on individual generative factors (e.g., day/night, high/low traffic) to isolate variations.
• Train separate β-VAEs on each partition to learn disentangled, compact latent representations where specific latent variables encode changes in the target generative factor.
• Select the most informative latent variable per factor using reconstruction error and disentanglement metrics, ensuring sensitivity to the target factor’s variation.
• Set a threshold τ as the 75th percentile of KL-divergence values from training samples to minimize false positives.
• During inference, compute the KL-divergence between the test image’s latent distribution and the training distribution; if above τ, flag as OOD.
• Use a three-phase pipeline: (1) dataset partitioning and hyperparameter tuning, (2) β-VAE training and latent variable selection, (3) real-time OOD detection using thresholded KL-divergence.

Experimental results

Research questions

• RQ1 Can the latent space of β-VAE reliably detect distributional shifts in specific generative factors (e.g., time-of-day, traffic, pedestrians) in multi-label datasets?
• RQ2 How can one-class OOD detection be adapted to multi-label datasets where labels are not mutually exclusive?
• RQ3 What hyperparameter configuration (β and latent dimension) of β-VAE yields optimal disentanglement for OOD detection on real-world autonomous driving data?
• RQ4 How effective is KL-divergence in the learned latent space as a metric for detecting OOD samples across different generative factors?
• RQ5 Can a single threshold τ be used effectively across diverse scene complexities without excessive false positives?

Key findings

• For the time-of-day factor, the β-VAE detector with β=1.8 and nLatent=30 achieved 99% in-distribution detection and 95% OOD detection on test sets.
• For the traffic density factor, the detector with β=1.6 and nLatent=30 achieved 90% in-distribution and 74% OOD detection, indicating moderate performance on subtle variations.
• For the pedestrian factor, the detector achieved 100% in-distribution and 100% OOD detection, demonstrating high sensitivity to this factor’s presence/absence.
• The 75th percentile threshold (τ75) was optimal for minimizing false positives while maintaining high detection rates across all partitions.
• Performance degraded in scenes with high background complexity, where background features were encoded across all latent variables, making factor-specific disentanglement difficult.
• The method is robust to scene complexity when the generative factor is clearly distinguishable (e.g., day/night), but struggles with subtle, context-dependent factors like traffic density.