[Paper Review] FIGR: Few-shot Image Generation with Reptile

Louis Clouâtre, Marc Demers|arXiv (Cornell University)|Jan 8, 2019

Generative Adversarial Networks and Image Synthesis15 references60 citations

TL;DR

FIGR meta-trains a GAN with Reptile to enable few-shot image generation from as few as 4 samples on MNIST and Omniglot, and introduces FIGR-8, a large few-shot benchmark dataset.

ABSTRACT

Generative Adversarial Networks (GAN) boast impressive capacity to generate realistic images. However, like much of the field of deep learning, they require an inordinate amount of data to produce results, thereby limiting their usefulness in generating novelty. In the same vein, recent advances in meta-learning have opened the door to many few-shot learning applications. In the present work, we propose Few-shot Image Generation using Reptile (FIGR), a GAN meta-trained with Reptile. Our model successfully generates novel images on both MNIST and Omniglot with as little as 4 images from an unseen class. We further contribute FIGR-8, a new dataset for few-shot image generation, which contains 1,548,944 icons categorized in over 18,409 classes. Trained on FIGR-8, initial results show that our model can generalize to more advanced concepts (such as "bird" and "knife") from as few as 8 samples from a previously unseen class of images and as little as 10 training steps through those 8 images. This work demonstrates the potential of training a GAN for few-shot image generation and aims to set a new benchmark for future work in the domain.

Motivation & Objective

• Define the problem of few-shot image generation and motivate meta-learning for GANs.
• Propose FIGR, a GAN meta-training framework using Reptile to adapt to unseen classes with minimal data.
• Introduce FIGR-8, a large-scale dataset to benchmark few-shot image generation.
• Demonstrate that FIGR can generate novel images from unseen classes with very small support sets (as few as 4 samples).

Proposed method

• Use Wasserstein loss with gradient penalty for GAN training.
• Meta-train both generator and discriminator using the Reptile algorithm with inner SGD updates on task-specific data.
• Adopt an outer loop update that moves initial weights toward task-adapted weights (Phi_d, Phi_g) via W_tau, enabling rapid adaptation.
• Apply inner loop with K gradient steps on a given task using n samples from that task’s class.
• After meta-training, perform few-shot generation by further updating generator/discriminator with small iterations on a new class.
• Experiment with MNIST, Omniglot, and FIGR-8 datasets to evaluate unseen-class generation.

Experimental results

Research questions

• RQ1 Can a GAN be trained via meta-learning to generate images from unseen classes with only a few examples?
• RQ2 How does Reptile-based meta-training enable rapid adaptation of GANs for few-shot image generation?
• RQ3 What is the performance of FIGR on standard few-shot benchmarks (MNIST, Omniglot) and on the large-scale FIGR-8 dataset?
• RQ4 Does the approach work with grayscale images and without external memory or lengthy inference?

Key findings

• FIGR can generate images from unseen MNIST digits with as few as 4 samples.
• On Omniglot, FIGR demonstrates rapid adaptation and reasonable generation quality with small support sets.
• FIGR-8 contains 1,548,944 images across 18,409 classes and provides a challenging benchmark for few-shot generation.
• Preliminary results on FIGR-8 show the model can learn concepts like bird-like shapes from 8 samples and 10 training steps.
• The approach does not require external memory or long sequential inference and uses a GAN framework with Reptile meta-training.
• Training with Wasserstein loss and gradient penalty yields stable performance across datasets.