[Paper Review] Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks

Alec Radford, Luke Metz+1 more|arXiv (Cornell University)|Nov 19, 2015

Generative Adversarial Networks and Image SynthesisComputer Science7,641 citations

TL;DR

The paper proposes DCGANs with architectural constraints that stabilize training and yield useful unsupervised representations, which can be repurposed for supervised tasks like image classification (e.g., CIFAR-10, SVHN). It also analyzes learned features, visualizes filters, and demonstrates latent-space arithmetic.

ABSTRACT

In recent years, supervised learning with convolutional networks (CNNs) has seen huge adoption in computer vision applications. Comparatively, unsupervised learning with CNNs has received less attention. In this work we hope to help bridge the gap between the success of CNNs for supervised learning and unsupervised learning. We introduce a class of CNNs called deep convolutional generative adversarial networks (DCGANs), that have certain architectural constraints, and demonstrate that they are a strong candidate for unsupervised learning. Training on various image datasets, we show convincing evidence that our deep convolutional adversarial pair learns a hierarchy of representations from object parts to scenes in both the generator and discriminator. Additionally, we use the learned features for novel tasks - demonstrating their applicability as general image representations.

Motivation & Objective

• Bridge the gap between supervised CNN success and unsupervised learning by proposing stable DCGAN architectures.
• Show that discriminator and generator learn hierarchical, interpretable representations.
• Demonstrate utility of learned features for downstream supervised tasks and visualize latent-space structure.
• Explore vector arithmetic and manipulation of generated samples to reveal learned semantics.

Proposed method

• Adopt architectural constraints to stabilize GAN training (all convolutional nets, no pooling, no fully connected top, global pooling compromise).
• Apply Batch Normalization with layer-wise exceptions to stabilize deep generators and discriminators.
• Use ReLU in the generator (output with Tanh) and LeakyReLU in the discriminator.
• Train with Adam optimizer on 128-sized mini-batches; initialize from zero-mean Gaussian; tuned learning rate and momentum (lr=0.0002, beta1=0.5).
• Train on LSUN, ImageNet-1k, and a Faces dataset with minimal preprocessing (scaled to [-1,1]); report qualitative and quantitative evaluations.
• Investigate internal representations via latent-space walking, guided backpropagation of discriminator features, and vector arithmetic in the latent space (Z).

Experimental results

Research questions

• RQ1 Can DCGAN architectures learn a coherent hierarchy of representations in generator and discriminator?
• RQ2 Are features learned by DCGANs transferable to downstream supervised tasks without labeled GAN supervision?
• RQ3 What qualitative properties (filters, latent space structure) do DCGANs acquire, and can latent vectors be manipulated to change semantic attributes?
• RQ4 Do DCGANs trained on large unlabeled datasets produce interpretable visual features and controllable generations?

Key findings

• DCGANs trained with the proposed architectural constraints allow stable training and scaling to higher resolution/depth across datasets (LSUN, ImageNet-1k, Faces).
• Discriminator features achieve competitive unsupervised representations used for supervised tasks; on CIFAR-10, DCGAN features plus linear SVM reach 82.8% accuracy.
• On SVHN with 1000 labeled examples, DCGAN features yield 22.48% test error, outperforming several baselines and a purely supervised CNN with the same architecture (28.87% error).
• Visualizations show learned filters activating on semantically meaningful bedroom components (beds, windows); latent space walking yields smooth, semantically meaningful image transitions (e.g., window appearance, bed presence).
• Vector arithmetic in the latent space demonstrates interpretable manipulations, including pose changes for faces and object-level changes (e.g., removing windows alters scene composition).
• The generator exhibits object-level disentanglement such that removing certain object-related filters (e.g., windows) alters generations while maintaining overall scene structure.