[Paper Review] SmoothGrad: removing noise by adding noise

Daniel Smilkov, Nikhil Thorat+3 more|arXiv (Cornell University)|Jun 12, 2017

Explainable Artificial Intelligence (XAI)3 references754 citations

TL;DR

SmoothGrad averages gradients over noisy copies of the input to produce sharper, more coherent sensitivity maps for image classifiers. Training with noise further enhances de-noising when combined with inference-time noise.

ABSTRACT

Explaining the output of a deep network remains a challenge. In the case of an image classifier, one type of explanation is to identify pixels that strongly influence the final decision. A starting point for this strategy is the gradient of the class score function with respect to the input image. This gradient can be interpreted as a sensitivity map, and there are several techniques that elaborate on this basic idea. This paper makes two contributions: it introduces SmoothGrad, a simple method that can help visually sharpen gradient-based sensitivity maps, and it discusses lessons in the visualization of these maps. We publish the code for our experiments and a website with our results.

Motivation & Objective

• Motivation to interpret deep neural networks and identify influential pixels for classification decisions.
• Address visual noise in gradient-based sensitivity maps and improve interpretability.
• Explore the benefits of smoothing gradients and combining with training-time noise.
• Provide evidence that SmoothGrad yields more visually coherent and discriminative maps.

Proposed method

• Define sensitivity maps as gradients of class scores with respect to input pixels.
• Introduce SmoothGrad by averaging gradients over inputs perturbed with Gaussian noise: M̂c(x) = (1/n) sumk Mc(x + N(0, σ^2)).
• Analyze the effect of noise level σ and sample size n on map sharpness and coherence.
• Compare SmoothGrad to vanilla gradients and other gradient-based methods (Integrated Gradients, Guided BackProp).
• Examine effects of applying noise during training versus inference.
• Offer visualization and combination strategies to enhance map interpretability.

Experimental results

Research questions

• RQ1 Does averaging gradients over noisy inputs reduce visual noise in saliency maps?
• RQ2 How do noise level σ and sample size n affect the quality of sensitivity maps?
• RQ3 Is SmoothGrad superior to vanilla gradients and other gradient-based methods in visual coherence and discriminativity?
• RQ4 Can SmoothGrad be effectively combined with other attribution methods?
• RQ5 Does training with noise further sharpen sensitivity maps, and is there an additive benefit when combined with inference-time noise?

Key findings

• SmoothGrad yields more visually coherent sensitivity maps than vanilla gradients and some baselines.
• A 10–20% noise level balances sharpness and structure for ImageNet images; results depend on input.
• Increasing the sample size n smooths the gradient, with diminishing returns after about n = 50.
• Combining SmoothGrad with Integrated Gradients or Guided BackProp improves visual coherence.
• Adding noise during training also sharpens sensitivity maps, with additive benefits when used with inference-time noise.