[Paper Review] Synthesizing the preferred inputs for neurons in neural networks via deep generator networks

Anh Son Nguyen, Alexey Dosovitskiy+3 more|arXiv (Cornell University)|May 30, 2016

Advanced Neural Network Applications24 references256 citations

TL;DR

The paper introduces DGN-AM, a method that uses a deep image generator prior to activation-maximize neurons in DNNs, producing realistic and interpretable visualizations that reveal learned features across architectures and datasets.

ABSTRACT

Deep neural networks (DNNs) have demonstrated state-of-the-art results on many pattern recognition tasks, especially vision classification problems. Understanding the inner workings of such computational brains is both fascinating basic science that is interesting in its own right - similar to why we study the human brain - and will enable researchers to further improve DNNs. One path to understanding how a neural network functions internally is to study what each of its neurons has learned to detect. One such method is called activation maximization (AM), which synthesizes an input (e.g. an image) that highly activates a neuron. Here we dramatically improve the qualitative state of the art of activation maximization by harnessing a powerful, learned prior: a deep generator network (DGN). The algorithm (1) generates qualitatively state-of-the-art synthetic images that look almost real, (2) reveals the features learned by each neuron in an interpretable way, (3) generalizes well to new datasets and somewhat well to different network architectures without requiring the prior to be relearned, and (4) can be considered as a high-quality generative method (in this case, by generating novel, creative, interesting, recognizable images).

Motivation & Objective

• Clarify what individual neurons in DNNs have learned to detect by synthesizing preferred inputs.
• Improve visualization realism and interpretability over priorAM methods by using a learned natural image prior.
• Demonstrate generalization of the learned prior across datasets and architectures, and assess limitations.
• Explore how the method reflects neuron-specific features versus prior-induced structure, and discuss broader applications.

Proposed method

• Use a deep image generator network (G) trained to invert intermediate features of a fixed encoder (E) to form a strong natural image prior.
• Optimize in the DGN input code y to maximize the target neuron activation: y^l^ = argmax_y^l (Phi_h(G_l(y^l)) - lambda*||y^l||) with L2 regularization (lambda ≈ 0.005).
• Clip optimized codes to the activation range [0, 3*sigma] per neuron to regularize the prior and improve image quality.
• Compare priors trained from different encoder layers (l in {3,5,6,7}) and select fc6-based G_6 as default for best visualizations.
• Assess generalization by visualizing neurons in networks trained on different datasets (MIT Places, action recognition) and different architectures (GoogLeNet, ResNet) using the same G_6 prior.
• Evaluate whether synthesized visuals resemble training data classes and whether the method memorizes training images by nearest-neighbor checks in deep feature spaces.

Experimental results

Research questions

• RQ1 Can a deep generator network prior enable qualitatively state-of-the-art activation maximization visualizations of neurons in DNNs?
• RQ2 Does the learned prior generalize across datasets and network architectures without retraining the generator?
• RQ3 How does visualization quality change when visualizing different architectures or hidden neurons?
• RQ4 Do the synthesized images reflect neuron-specific features or predominantly reflect the prior?
• RQ5 What are the broader applications and limitations of DGN-AM for interpreting DNNs?

Key findings

• DGN-AM produces highly realistic and interpretable images for activated neurons, outperforming previous activation maximization methods in visual quality.
• The fc6-based generator G_6 provides the best visualizations among layers tested (l ∈ {3,5,6,7}).
• The ImageNet-trained prior generalizes to other natural-image datasets (e.g., MIT Places) when visualizing networks of the same encoder architecture, with varying image sharpness depending on architecture differences.
• Using the same DGN prior, visualizations degrade as the target network architecture diverges from the encoder used to train the generator (best with E identical to Phi).
• DGN-AM can visualize hidden neurons across layers, often aligning with other visualization methods and producing more realistic color/texture visuals.
• The method remains informative for multi-modal or non-canonical neuron features, indicating that some neurons learn diverse features beyond single canonical visuals.