[Paper Review] Using KL-divergence to focus Deep Visual Explanation

Housam Khalifa Bashier Babiker, Randy Goebel|arXiv (Cornell University)|Nov 17, 2017

Explainable Artificial Intelligence (XAI)9 references3 citations

TL;DR

This paper proposes a novel method to explain deep convolutional neural network predictions by using Kullback-Leibler (KL) divergence to focus on salient image regions that influence classification. By computing the KL-divergence gradient between predicted and ground-truth class scores, normalizing it, and applying it as a weighting map over feature maps, the approach generates attention heatmaps that highlight discriminative pixels, demonstrating improved interpretability on VGG-16 and AlexNet models with clearer localization than baseline methods.

ABSTRACT

We present a method for explaining the image classification predictions of deep convolution neural networks, by highlighting the pixels in the image which influence the final class prediction. Our method requires the identification of a heuristic method to select parameters hypothesized to be most relevant in this prediction, and here we use Kullback-Leibler divergence to provide this focus. Overall, our approach helps in understanding and interpreting deep network predictions and we hope contributes to a foundation for such understanding of deep learning networks. In this brief paper, our experiments evaluate the performance of two popular networks in this context of interpretability.

Motivation & Objective

• To address the interpretability challenge in deep convolutional neural networks, especially in high-stakes domains like medical imaging.
• To develop a method that identifies and highlights the most influential pixels in an image for a given classification decision.
• To improve visualization quality by focusing on discriminative regions using KL-divergence as a metric for relevance.
• To compare the performance of different network architectures (VGG-16 and AlexNet) in generating interpretable explanations.

Proposed method

• Compute raw class scores from the final layer of a pre-trained CNN (e.g., VGG-16) and use them to estimate joint probabilities via a Gaussian kernel-based similarity measure.
• Estimate pairwise affinities between class scores using perplexity to model the ground truth distribution.
• Compute the KL-divergence gradient between the predicted and ground-truth distributions to identify relevant neurons and features.
• Normalize the KL-gradient using z-score normalization (zero mean, unit variance) to obtain attention weights α.
• Apply the normalized weights α as a spatial attention map by element-wise multiplication with feature maps, then sum across channels to produce a saliency map.
• Normalize the final saliency map for visualization, producing a heatmap that highlights pixels most influential in the classification decision.

Experimental results

Research questions

• RQ1 How can KL-divergence be used to focus visual explanations on the most relevant image regions in deep neural networks?
• RQ2 Does the quality of visual explanations correlate with the accuracy of the underlying deep learning model?
• RQ3 How does the proposed KL-based method compare to existing explanation techniques like guided backpropagation and Grad-CAM in terms of localization precision?
• RQ4 Can the KL-divergence gradient effectively capture discriminative features across diverse image categories?

Key findings

• The proposed method produces sharper and more focused saliency maps compared to baseline methods like guided backpropagation and Grad-CAM, especially in complex scenes.
• VGG-16, being more accurate than AlexNet, generated more localized and precise attention maps, indicating a direct correlation between model accuracy and explanation quality.
• The KL-divergence gradient effectively identifies discriminative regions such as the face of a cockatoo, the lens of a racket, and the sign of a traffic light, aligning with human intuition.
• Normalization of the KL-gradient using z-score transformation improved the stability and interpretability of the resulting attention maps.
• The method successfully visualizes evidence for classification decisions across diverse COCO dataset images, demonstrating robustness across object categories.
• The algorithm is generalizable and can be extended to other domains such as speech and natural language processing for interpretability.