[Paper Review] Visualizing and Understanding Convolutional Networks

Matthew D. Zeiler, Rob Fergus|arXiv (Cornell University)|Nov 12, 2013

Explainable Artificial Intelligence (XAI)17 references448 citations

TL;DR

This paper introduces a deconvolutional network (deconvnet) visualization technique that maps intermediate feature activations back to input pixel space, revealing how convolutional networks learn hierarchical, class-discriminative features. Using this method, the authors diagnose and improve model architecture, achieving superior ImageNet top-1 error (13.7%) compared to Krizhevsky et al.'s 16.4%, and demonstrate strong generalization to Caltech-101 and Caltech-256 by fine-tuning only the softmax classifier.

ABSTRACT

Large Convolutional Network models have recently demonstrated impressive classification performance on the ImageNet benchmark. However there is no clear understanding of why they perform so well, or how they might be improved. In this paper we address both issues. We introduce a novel visualization technique that gives insight into the function of intermediate feature layers and the operation of the classifier. We also perform an ablation study to discover the performance contribution from different model layers. This enables us to find model architectures that outperform Krizhevsky \etal on the ImageNet classification benchmark. We show our ImageNet model generalizes well to other datasets: when the softmax classifier is retrained, it convincingly beats the current state-of-the-art results on Caltech-101 and Caltech-256 datasets.

Motivation & Objective

• To develop a diagnostic tool that reveals the internal functioning of deep convolutional networks, which are often treated as black boxes.
• To understand why large-scale convolutional networks achieve state-of-the-art performance on image classification benchmarks.
• To improve model architecture through visualization-guided diagnostics, leading to better performance on ImageNet.
• To evaluate the generalization capability of ImageNet-pretrained features on other datasets like Caltech-101 and Caltech-256.
• To analyze the contribution of individual layers and architectural components to overall model performance via ablation.

Proposed method

• A multi-layer deconvolutional network (deconvnet) is used to project feature activations from intermediate layers back to the input pixel space, reconstructing the stimuli that most strongly activate each feature map.
• The deconvnet performs transposed convolutions and ReLU activations in reverse order of the original convolutional network, enabling top-down visualization of feature responses.
• Sensitivity analysis via occlusion is applied to identify which image regions are most critical for classification, revealing local structure dependence.
• Ablation studies are conducted by systematically removing or modifying layers and components (e.g., pooling, normalization) to assess their impact on performance.
• Transfer learning is evaluated by retraining only the final softmax classifier on features from the ImageNet-pretrained model for Caltech-101, Caltech-256, and PASCAL 2012.
• Linear SVMs and softmax classifiers are trained on features from different layers to assess the discriminative power of hierarchical features.

Experimental results

Research questions

• RQ1 What kind of visual patterns activate specific feature maps in deep convolutional networks, and how do they evolve across layers?
• RQ2 How can visualization techniques be used to diagnose and improve convolutional network architecture?
• RQ3 To what extent do features learned on ImageNet generalize to other image classification benchmarks?
• RQ4 Which architectural components (e.g., pooling, normalization) are most critical for performance, and how do they contribute?
• RQ5 How sensitive is the model to local image structures, and does it rely on global scene context?

Key findings

• The deconvolutional visualization technique reveals that feature maps in deeper layers learn increasingly complex, class-discriminative, and compositional patterns, such as edges, textures, and object parts.
• Using visualization-guided architecture search, the authors achieve a top-1 error rate of 13.7% on ImageNet, outperforming Krizhevsky et al.'s 16.4%.
• On Caltech-101 and Caltech-256, fine-tuning only the softmax classifier on ImageNet-pretrained features achieves state-of-the-art performance, with 85.4% accuracy on Caltech-101 and 72.6% on Caltech-256.
• The ablation study shows that network depth is more critical than any individual component, with performance dropping significantly when the network is too shallow.
• Occlusion experiments reveal that the model is highly sensitive to local image structures, not just global scene context, indicating reliance on fine-grained features for classification.
• Features from deeper layers (e.g., layer 5 and 7) contain significantly more discriminative information than shallow layers, as shown by progressively higher SVM and softmax accuracy with increasing depth.