[Paper Review] How convolutional neural network see the world - A survey of convolutional neural network visualization methods

Zhuwei Qin, Funxun Yu+2 more|arXiv (Cornell University)|Apr 30, 2018

Anomaly Detection Techniques and Applications47 references48 citations

TL;DR

A comprehensive survey of CNN visualization methods (Activation Maximization, DeconvNet, Network Inversion, Network Dissection) that interpret CNN internals and semantics, with discussion of motivations, algorithms, experiments, and applications.

ABSTRACT

Nowadays, the Convolutional Neural Networks (CNNs) have achieved impressive performance on many computer vision related tasks, such as object detection, image recognition, image retrieval, etc. These achievements benefit from the CNNs outstanding capability to learn the input features with deep layers of neuron structures and iterative training process. However, these learned features are hard to identify and interpret from a human vision perspective, causing a lack of understanding of the CNNs internal working mechanism. To improve the CNN interpretability, the CNN visualization is well utilized as a qualitative analysis method, which translates the internal features into visually perceptible patterns. And many CNN visualization works have been proposed in the literature to interpret the CNN in perspectives of network structure, operation, and semantic concept. In this paper, we expect to provide a comprehensive survey of several representative CNN visualization methods, including Activation Maximization, Network Inversion, Deconvolutional Neural Networks (DeconvNet), and Network Dissection based visualization. These methods are presented in terms of motivations, algorithms, and experiment results. Based on these visualization methods, we also discuss their practical applications to demonstrate the significance of the CNN interpretability in areas of network design, optimization, security enhancement, etc.

Motivation & Objective

• Clarify the motivation for CNN visualization and interpretability.
• Summarize four representative visualization methods and their core ideas.
• Compare methods in terms of goals, algorithms, and observed results.
• Discuss practical applications of CNN visualization in design, optimization, and security.

Proposed method

• Describe Activation Maximization (AM) and its objective to synthesize inputs that maximize neuron activations.
• Explain AM enhancements: regularization, and Deep Generative Network Activation Maximization (DGN-AM).
• Present Deconvolutional Network (DeconvNet) visualization and its reversed-layer propagation to project feature maps back to input space.
• Discuss Network Inversion to reconstruct inputs from layer activations.
• Introduce Network Dissection to semantically label neurons using a heterogeneous dataset.
• Summarize experimental setups on architectures like CaffeNet/ImageNet to illustrate learned features.

Experimental results

Research questions

• RQ1 What visual patterns activate individual neurons and layers in CNNs?
• RQ2 How do different visualization approaches relate internal CNN features to human-interpretable patterns or semantics?
• RQ3 What regularization or generative techniques improve the interpretability of visualizations?
• RQ4 What practical insights do visualization methods provide for CNN design, optimization, and security?

Key findings

• Activation Maximization reveals hierarchical and interpretable features such as edges, shapes, and objects, with patterns becoming more complex in deeper layers.
• Regularization and generative approaches (DGN-AM) improve the realism and interpretability of synthesized patterns in higher layers.
• DeconvNet visualizations provide explicit, image-level patterns that show which input features trigger specific neurons across layers.
• Network Inversion demonstrates what input information is preserved at each layer by reconstructing inputs from feature maps.
• Network Dissection enables semantic labeling of neurons, linking units to predefined visual concepts such as objects, parts, materials, textures, colors, and scenes.
• Visualizations reveal that CNNs often learn localized, pattern-specific features and hierarchical feature extraction akin to the visual cortex.