[Paper Review] Semi-Supervised Learning with Context-Conditional Generative Adversarial Networks

Emily Denton, Sam Gross+1 more|arXiv (Cornell University)|Nov 19, 2016

Generative Adversarial Networks and Image SynthesisComputer Science26 references93 citations

TL;DR

This paper proposes a semi-supervised learning method for image classification using a context-conditional GAN that reconstructs missing image patches via in-painting. By training a generator to fill in random holes and a discriminator to distinguish real from generated images, the approach regularizes supervised learning, enabling effective training of large VGG-style networks and achieving state-of-the-art or competitive performance on STL-10 and PASCAL datasets.

ABSTRACT

We introduce a simple semi-supervised learning approach for images based on in-painting using an adversarial loss. Images with random patches removed are presented to a generator whose task is to fill in the hole, based on the surrounding pixels. The in-painted images are then presented to a discriminator network that judges if they are real (unaltered training images) or not. This task acts as a regularizer for standard supervised training of the discriminator. Using our approach we are able to directly train large VGG-style networks in a semi-supervised fashion. We evaluate on STL-10 and PASCAL datasets, where our approach obtains performance comparable or superior to existing methods.

Motivation & Objective

• To develop a simple yet effective semi-supervised learning framework for image classification using adversarial training.
• To leverage unlabeled data through image in-painting as a self-supervised regularizer.
• To enable end-to-end training of deep VGG-style networks in a semi-supervised setting.
• To improve generalization and performance on limited labeled data by exploiting structural consistency in images.

Proposed method

• A generator network is trained to reconstruct missing patches in images by predicting pixel values based on surrounding context.
• The generator's output is fed into a discriminator that classifies images as real (original) or fake (in-painted).
• The adversarial loss from the discriminator provides a regularizing signal during supervised training of the classification head.
• The model jointly optimizes for both classification accuracy on labeled data and adversarial realism of in-painted regions.
• The approach uses a standard VGG-style architecture for both generator and discriminator, enabling scalability to deeper networks.

Experimental results

Research questions

• RQ1 Can in-painting via a conditional GAN serve as an effective regularizer for semi-supervised image classification?
• RQ2 How well can a context-conditional GAN-based approach generalize when only a small number of labeled examples are available?
• RQ3 Does the adversarial in-painting loss improve performance compared to standard semi-supervised methods on benchmark datasets?
• RQ4 Can large VGG-style networks be effectively trained in a semi-supervised manner using this framework?

Key findings

• The proposed method achieves performance comparable to or better than existing state-of-the-art methods on the STL-10 dataset.
• The approach enables direct training of large VGG-style networks in a semi-supervised setting, which is challenging with traditional methods.
• The in-painting task acts as an effective regularizer, improving generalization despite limited labeled data.
• The model demonstrates strong performance on the PASCAL dataset, indicating robustness across different image classification benchmarks.