[Paper Review] Data Augmentation Generative Adversarial Networks

Antreas Antoniou, Amos Storkey+1 more|arXiv (Cornell University)|Nov 12, 2017

Domain Adaptation and Few-Shot Learning20 references806 citations

TL;DR

DAGAN learns to generate within-class data augmentations from a source domain using a conditional GAN to improve low-data target domains, boosting vanilla classifiers and few-shot learning methods; tested on Omniglot, EMNIST, and VGG-Face with notable accuracy gains.

ABSTRACT

Effective training of neural networks requires much data. In the low-data regime, parameters are underdetermined, and learnt networks generalise poorly. Data Augmentation alleviates this by using existing data more effectively. However standard data augmentation produces only limited plausible alternative data. Given there is potential to generate a much broader set of augmentations, we design and train a generative model to do data augmentation. The model, based on image conditional Generative Adversarial Networks, takes data from a source domain and learns to take any data item and generalise it to generate other within-class data items. As this generative process does not depend on the classes themselves, it can be applied to novel unseen classes of data. We show that a Data Augmentation Generative Adversarial Network (DAGAN) augments standard vanilla classifiers well. We also show a DAGAN can enhance few-shot learning systems such as Matching Networks. We demonstrate these approaches on Omniglot, on EMNIST having learnt the DAGAN on Omniglot, and VGG-Face data. In our experiments we can see over 13% increase in accuracy in the low-data regime experiments in Omniglot (from 69% to 82%), EMNIST (73.9% to 76%) and VGG-Face (4.5% to 12%); in Matching Networks for Omniglot we observe an increase of 0.5% (from 96.9% to 97.4%) and an increase of 1.8% in EMNIST (from 59.5% to 61.3%).

Motivation & Objective

• Motivate improved generalization in low-data regimes by learning richer data augmentation strategies.
• Propose a Data Augmentation GAN (DAGAN) that learns cross-class, within-class augmentations from a source domain.
• Demonstrate that DAGAN augmentations improve vanilla classifiers and few-shot learning systems.
• Show cross-domain applicability by transferring augmentations from Omniglot to EMNIST and VGG-Face datasets.

Proposed method

• Introduce a DAGAN architecture that combines a UNet/ResNet style generator (UResNet) with a DenseNet discriminator.
• Use an encoder g(x) to obtain a representation r, concatenate a latent z ~ N(0,I), and decode to generate augmentation images x = f(z,r).
• Train the DAGAN in a source domain using an improved Wasserstein GAN objective with a critic that compares same-class real pairs to GAN-generated augmentations.
• Provide the original input x to the discriminator to ensure the model learns cross-class generalizable augmentations without relying on class labels.
• Train a DenseNet classifier on real data plus DAGAN augmentations, using a real/fake label to guide the network to weigh true versus generated data.

Experimental results

Research questions

• RQ1 Can a learned data augmentation model generalize across unseen classes in low-data regimes?
• RQ2 Do DAGAN-generated augmentations improve classification performance in vanilla and few-shot learning settings across diverse datasets?
• RQ3 Is meta-learning of augmentation strategies beneficial for one-shot learning compared to standard augmentation and other meta-learning approaches?
• RQ4 Can augmentations learned in a source domain transfer effectively to target domains with substantial dataset shifts?

Key findings

• DAGAN augmentation improves vanilla classifier accuracy across all target-domain experiments.
• Omniglot: accuracy increases from 69.0% (standard) to 82.1% (DAGAN augmented) with 5 samples per class.
• EMNIST: accuracy increases from 73.9% (standard) to 76.1% (DAGAN augmented) with 15 samples per class; up to 0. value increase with more samples.
• VGG-Face: accuracy increases from 4.5% (standard) to 12.0% (DAGAN augmented) with 5 samples per class.
• In one-shot matching networks, DAGAN augmentation yields improvements, e.g., Omniglot from 96.9% to 97.4% (+0.5%), and EMNIST from 59.5% to 61.3% (+1.8%).
• Table 1 shows vanilla classifier results with DAGAN augmentation improving test accuracy across Omniglot, EMNIST, and VGG-Face datasets.
• Table 2 situates DAGAN-augmented results against various one-shot learning methods on Omniglot, showing competitive gains.