[Paper Review] On the Importance of Gradients for Detecting Distributional Shifts in the Wild

Rui Huang, Andrew Geng+1 more|arXiv (Cornell University)|Oct 1, 2021

Anomaly Detection Techniques and ApplicationsComputer Science88 citations

TL;DR

GradNorm leverages gradient norms backpropagated from the KL divergence between softmax outputs and a uniform distribution to detect out-of-distribution inputs, achieving superior OOD detection on ImageNet benchmarks compared to prior output- and feature-based methods.

ABSTRACT

Detecting out-of-distribution (OOD) data has become a critical component in ensuring the safe deployment of machine learning models in the real world. Existing OOD detection approaches primarily rely on the output or feature space for deriving OOD scores, while largely overlooking information from the gradient space. In this paper, we present GradNorm, a simple and effective approach for detecting OOD inputs by utilizing information extracted from the gradient space. GradNorm directly employs the vector norm of gradients, backpropagated from the KL divergence between the softmax output and a uniform probability distribution. Our key idea is that the magnitude of gradients is higher for in-distribution (ID) data than that for OOD data, making it informative for OOD detection. GradNorm demonstrates superior performance, reducing the average FPR95 by up to 16.33% compared to the previous best method.

Motivation & Objective

• Motivate robust OOD detection for real-world deployments beyond in-distribution accuracy.
• Explore whether gradient space contains useful signals for distinguishing ID from OOD data.
• Propose a simple, training-free method (GradNorm) that uses gradient magnitudes as an OOD score.
• Analyze how gradient space information combines feature and output signals for improved separability.

Proposed method

• Compute gradients of the KL divergence between the softmax output and a uniform distribution with respect to network parameters.
• Backpropagate these gradients to obtain a gradient vector and use its vector norm as the OOD score.
• Use the last fully connected layer's weights as the gradient source for efficiency and strong performance.
• Operate in a label-agnostic and OOD-agnostic setting, requiring no ground-truth labels for backpropagation.
• Investigate the effect of different Lp norms (favoring L1) and temperature T on GradNorm performance.

Experimental results

Research questions

• RQ1 Can gradient space provide discriminative signals for ID vs. OOD data beyond activation or output spaces?
• RQ2 What gradient source (which network layer/parameters) yields the best OOD detection performance and computational efficiency?
• RQ3 Which loss formulation and vector norm maximize the separability between ID and OOD when using gradient-based scores?
• RQ4 How do architectural choices and temperature scaling affect the effectiveness of gradient-based OOD detection?

Key findings

• GradNorm reduces average FPR95 by up to 16.33% compared to the previous best method on ImageNet benchmarks.
• Gradient norms derived from the last FC layer offer the best trade-off between performance and computation.
• Using the uniform target for KL divergence (rather than a one-hot target) yields stronger ID vs. OOD separation, with GradNorm outperforming the one-hot variant by substantial margins.
• The L1 norm of gradients consistently outperforms other norms (including higher-order norms and infinity-norm) for GradNorm.
• GradNorm captures joint information from both feature and output spaces, leading to stronger separability than using feature or output information alone.