[Paper Review] Informative Class Activation Maps

Zhenyue Qin, Dongwoo Kim+1 more|arXiv (Cornell University)|Jun 19, 2021

Domain Adaptation and Few-Shot Learning12 references4 citations

TL;DR

This paper proposes Informative Class Activation Maps (infoCAM), a novel method that leverages information theory to identify the most informative image regions for classification by maximizing the pointwise mutual information (PMI) difference between the true label and other labels. infoCAM outperforms traditional CAMs in weakly supervised object localization, achieving state-of-the-art performance on Tiny-ImageNet and producing more accurate, object-centric localization maps by focusing on discriminative features beyond just the most activated regions.

ABSTRACT

We study how to evaluate the quantitative information content of a region within an image for a particular label. To this end, we bridge class activation maps with information theory. We develop an informative class activation map (infoCAM). Given a classification task, infoCAM depict how to accumulate information of partial regions to that of the entire image toward a label. Thus, we can utilise infoCAM to locate the most informative features for a label. When applied to an image classification task, infoCAM performs better than the traditional classification map in the weakly supervised object localisation task. We achieve state-of-the-art results on Tiny-ImageNet.

Motivation & Objective

• To formalize class activation maps using information theory to quantify the information content of image regions for a given label.
• To address the limitation of traditional CAMs, which may highlight non-discriminative or partial features, by identifying regions with maximal mutual information to the true label.
• To improve weakly supervised object localization (WSOL) by generating more precise and complete object bounding boxes.
• To provide a theoretical reinterpretation of CNN classifiers as mutual information evaluators, enhancing model interpretability.

Proposed method

• infoCAM computes a region-specific intensity score based on the difference in pointwise mutual information (PMI) between the true label and all other labels.
• The core equation for infoCAM intensity is $ M_{y}^{ ext{Diff}}(R) = \sum_{(a,b)\in R} w^{y}g(a,b) - \frac{1}{M-1}\sum_{y'\neq y} w^{y'}g(a,b) $, which emphasizes regions that strongly support the true class over others.
• The method uses the final convolutional feature maps from a CNN and applies a threshold to select high-information regions for bounding box generation.
• infoCAM+ simplifies the computation by focusing only on the true label and the most unlikely label, maintaining performance while reducing complexity.
• The bounding box is formed by finding the largest connected region exceeding a threshold in the infoCAM map and then upscaling it to the original image resolution.
• The approach is evaluated using standard WSOL benchmarks, including CUB-200-2011 and Tiny-ImageNet, with ablation studies on multi-label localization using a double-digit MNIST dataset.

Experimental results

Research questions

• RQ1 Can information-theoretic principles improve the interpretability and accuracy of class activation maps in weakly supervised object localization?
• RQ2 How does the PMI difference between the true label and other labels help identify more discriminative image regions than standard CAMs?
• RQ3 Does infoCAM lead to more complete and precise localization of objects, especially when compared to CAMs that often miss parts of the object?
• RQ4 Can infoCAM generalize across different datasets and network architectures, including those with small image sizes like Tiny-ImageNet?
• RQ5 How does infoCAM perform in multi-label localization tasks, where multiple objects may be present in a single image?

Key findings

• infoCAM achieves state-of-the-art performance on the Tiny-ImageNet dataset for weakly supervised object localization, outperforming traditional CAMs.
• On CUB-200-2011, infoCAM improves localization accuracy over standard CAMs across multiple backbone networks, demonstrating consistent gains.
• In the multi-label double-digit MNIST experiment, infoCAM increases localization accuracy from 91% (CAM) to 98%, showing superior performance in complex localization scenarios.
• Visualizations confirm that infoCAM highlights entire objects (e.g., bird bodies), whereas CAMs often focus only on partial or less discriminative parts (e.g., bird heads).
• infoCAM+ achieves comparable performance to infoCAM with reduced computational cost, making it a practical alternative for real-world deployment.
• The use of ADL (Adversarial Distillation Loss) improves performance on CUB-200-2011 but degrades results on Tiny-ImageNet, suggesting sensitivity to image size and structure.