[Paper Review] Probabilistic Modeling of Deep Features for Out-of-Distribution and Adversarial Detection

Nilesh Ahuja, Ibrahima J. Ndiour+2 more|arXiv (Cornell University)|Sep 25, 2019

Adversarial Robustness in Machine Learning21 references37 citations

TL;DR

The paper proposes modeling the deep features of a trained DNN with class-conditional parametric distributions (Gaussian and Gaussian Mixture Models) to detect out-of-distribution and adversarial samples using per-layer log-likelihoods. It demonstrates improved OOD and adversarial detection on MNIST, CIFAR-10, and UCF101 video data, with analyses on dimensionality reduction and layer-wise effectiveness.

ABSTRACT

We present a principled approach for detecting out-of-distribution (OOD) and adversarial samples in deep neural networks. Our approach consists in modeling the outputs of the various layers (deep features) with parametric probability distributions once training is completed. At inference, the likelihoods of the deep features w.r.t the previously learnt distributions are calculated and used to derive uncertainty estimates that can discriminate in-distribution samples from OOD samples. We explore the use of two classes of multivariate distributions for modeling the deep features - Gaussian and Gaussian mixture - and study the trade-off between accuracy and computational complexity. We demonstrate benefits of our approach on image features by detecting OOD images and adversarially-generated images, using popular DNN architectures on MNIST and CIFAR10 datasets. We show that more precise modeling of the feature distributions result in significantly improved detection of OOD and adversarial samples; up to 12 percentage points in AUPR and AUROC metrics. We further show that our approach remains extremely effective when applied to video data and associated spatio-temporal features by detecting adversarial samples on activity classification tasks using UCF101 dataset, and the C3D network. To our knowledge, our methodology is the first one reported for reliably detecting white-box adversarial framing, a state-of-the-art adversarial attack for video classifiers.

Motivation & Objective

• Motivate reliable predictive uncertainty in DNNs for safety-critical domains.
• Propose a generative approach that models deep-feature distributions post-training.
• Evaluate how per-layer likelihoods enable OOD and adversarial detection without changing the original classifier.
• Analyze computational trade-offs and the impact of dimensionality reduction on parameter estimation.

Proposed method

• Fit class-conditional distributions to deep features from multiple layers using multivariate Gaussians (separate covariances) or Gaussian Mixture Models.
• Estimate parameters with maximum likelihood (means and covariances) or EM for GMMs; use BIC for component selection.
• Compute per-layer log-likelihoods for test samples and use them to detect OOD/adversarial samples and potentially classify.
• Use either Mahalanobis distance (tied covariance) or full log-likelihood depending on covariance assumptions.
• Apply dimensionality reduction (average pooling and PCA) to mitigate ill-conditioning and reduce computation.
• Evaluate with AUPR and AUROC metrics on MNIST and CIFAR-10, and extend to video with UCF101 and adversarial framing attacks.

Experimental results

Research questions

• RQ1 Can per-layer probabilistic modeling of deep features improve detection of OOD samples compared to baseline softmax-based methods?
• RQ2 Do Gaussian, separate-covariance Gaussian, and Gaussian Mixture Models offer different trade-offs in accuracy and computational cost for OOD/adversarial detection?
• RQ3 How do dimensionality reduction techniques affect parameter estimation and detection performance?
• RQ4 Is the approach effective on video data and against white-box and black-box adversarial framing attacks?

Key findings

• Modeling deep features with class-conditional distributions yields competitive in-distribution classification and improved OOD/adversarial detection (up to 12 percentage points in AUPR/AUROC in some cases).
• Gaussian with separate covariances and GMMs provide comparable detection performance; more general models improve robustness especially in earlier (inner) layers.
• Dimensionality reduction via PCA and average pooling alleviates ill-conditioning and enhances detection and classification performance.
• The approach is effective on video data (UCF101) and against white-box and black-box adversarial framing attacks, improving AUROC/AUPR over baselines.
• Layer-wise improvements in detection accuracy increase as one moves away from the final layer, indicating inner layers benefit more from richer distributions.
• Per-layer log-likelihoods can be used for classification with accuracy comparable to softmax on in-distribution data.