[Paper Review] Smooth Grad-CAM++: An Enhanced Inference Level Visualization Technique for Deep Convolutional Neural Network Models

Daniel Omeiza, Skyler Speakman+2 more|arXiv (Cornell University)|Aug 3, 2019

Explainable Artificial Intelligence (XAI)13 references158 citations

TL;DR

Smooth Grad-CAM++ combines SmoothGrad and Grad-CAM++ to provide sharper, better-localized visual explanations at inference time, including layer, feature map, and neuron-level visualization.

ABSTRACT

Gaining insight into how deep convolutional neural network models perform image classification and how to explain their outputs have been a concern to computer vision researchers and decision makers. These deep models are often referred to as black box due to low comprehension of their internal workings. As an effort to developing explainable deep learning models, several methods have been proposed such as finding gradients of class output with respect to input image (sensitivity maps), class activation map (CAM), and Gradient based Class Activation Maps (Grad-CAM). These methods under perform when localizing multiple occurrences of the same class and do not work for all CNNs. In addition, Grad-CAM does not capture the entire object in completeness when used on single object images, this affect performance on recognition tasks. With the intention to create an enhanced visual explanation in terms of visual sharpness, object localization and explaining multiple occurrences of objects in a single image, we present Smooth Grad-CAM++ \footnote{Simple demo: http://35.238.22.135:5000/}, a technique that combines methods from two other recent techniques---SMOOTHGRAD and Grad-CAM++. Our Smooth Grad-CAM++ technique provides the capability of either visualizing a layer, subset of feature maps, or subset of neurons within a feature map at each instance at the inference level (model prediction process). After experimenting with few images, Smooth Grad-CAM++ produced more visually sharp maps with better localization of objects in the given input images when compared with other methods.

Motivation & Objective

• MotivateExplainable AI by improving localization and visual sharpness of Grad-CAM-based explanations for CNNs.
• Enable visualization of not only entire feature maps but also subsets of neurons within a feature map.
• Provide an inference-time tool that can visualize layers, specific feature maps, and neuron coordinates for better debugging.
• Incorporate gradient smoothing to produce more complete and visually appealing saliency maps.

Proposed method

• Integrate smoothing from SmoothGrad into Grad-CAM++ by averaging gradients over multiple noisy input samples.
• Compute averaged first, second, and third-order partial derivatives across n noisy inputs to obtain alpha and W weights as in Grad-CAM++, adapted with gradient averages.
• Apply the averaged gradients in the Grad-CAM++ formulation to generate a final class-discriminative saliency map.
• Offer an API enabling visualization of a chosen layer, a subset of feature maps, and neurons within a region or coordinates in a feature map.
• Allow visualization at inference time without retraining, using a pre-trained model and a specified convolutional layer (e.g., VGG-16 last conv layer).
• Provide options to select region-based neuron visualization or coordinate-based neuron subsets for debugging purposes.

Experimental results

Research questions

• RQ1 Can SmoothGrad-inspired gradient averaging improve the visual sharpness and localization of Grad-CAM++ saliency maps?
• RQ2 Does Smooth Grad-CAM++ retain or improve object localization when multiple occurrences of the same class appear in an image?
• RQ3 Can users visualize not just whole feature maps but also subsets of neurons within a feature map at inference time?
• RQ4 How does the method perform across different layers and feature maps for CNNs without retraining?
• RQ5 What is the effect of noise level and number of samples on the quality of the saliency maps?

Key findings

• Smooth Grad-CAM++ yields visually sharper saliency maps compared to Grad-CAM and Grad-CAM++, with improved localization.
• The method captures larger portions of the object and better localization in example images.
• The technique enables visualization at the level of individual feature maps and subsets of neurons within a feature map.
• An API supports selecting layers, feature maps, and neuron regions for flexible, neuron-level explanations.
• Results suggest enhanced explainability at inference time without retraining, particularly for multiple object occurrences.