[Paper Review] GAN Augmentation: Augmenting Training Data using Generative Adversarial Networks

Christopher Bowles, Liang Chen+8 more|arXiv (Cornell University)|Oct 25, 2018

Generative Adversarial Networks and Image Synthesis18 references359 citations

TL;DR

The paper shows that GAN-derived synthetic data can improve brain image segmentation performance, especially when real labeled data are scarce, with Dice Similarity Coefficient gains of about 1–5 percentage points across CT and MR tasks.

ABSTRACT

One of the biggest issues facing the use of machine learning in medical imaging is the lack of availability of large, labelled datasets. The annotation of medical images is not only expensive and time consuming but also highly dependent on the availability of expert observers. The limited amount of training data can inhibit the performance of supervised machine learning algorithms which often need very large quantities of data on which to train to avoid overfitting. So far, much effort has been directed at extracting as much information as possible from what data is available. Generative Adversarial Networks (GANs) offer a novel way to unlock additional information from a dataset by generating synthetic samples with the appearance of real images. This paper demonstrates the feasibility of introducing GAN derived synthetic data to the training datasets in two brain segmentation tasks, leading to improvements in Dice Similarity Coefficient (DSC) of between 1 and 5 percentage points under different conditions, with the strongest effects seen fewer than ten training image stacks are available.

Motivation & Objective

• Motivate data augmentation in medical imaging where labeled data are expensive and scarce.
• Investigate whether GAN-generated synthetic patches can augment training data for segmentation networks.
• Evaluate the effect of GAN augmentation across different segmentation architectures and datasets.
• Assess interaction with traditional augmentation and determine data quantity thresholds for benefit.

Proposed method

• Use Progressive Growing GANs (PGGAN) to model the joint distribution of image patches and segmentation labels.
• Train GAN on 80k patches (real image + label) to learn the data manifold.
• Sample synthetic patches from the GAN and augment the 80k real patches for segmentation network training.
• Evaluate with three segmentation networks (UNet, UResNet, DeepMedic) on CT CSF and MR WMH tasks.
• Vary amount of real data and amount of synthetic data to study impact on Dice Similarity Coefficient (DSC).
• Compare GAN augmentation to rotation augmentation and combinations of augmentations.

Experimental results

Research questions

• RQ1 Does the segmentation network architecture affect the benefit of GAN augmentation?
• RQ2 How does GAN augmentation compare to rotation augmentation?
• RQ3 How does the amount of synthetic data influence performance?
• RQ4 How does the amount of available real data influence the benefit of GAN augmentation?
• RQ5 Does GAN augmentation generalize across different datasets and modalities (CT/MR)?

Key findings

• GAN augmentation yields modest but significant DSC improvements in many cases.
• Largest gains occur when real data are most limited; benefits taper only at very low data availability for MR in some scenarios.
• CT results show no observed degradation from synthetic data; MR results show a potential tipping point where too much synthetic data can slightly hurt performance.
• Adding GAN augmentation and traditional augmentation can yield greater gains than either alone, suggesting complementary effects.
• GANs generate synthetic images that differ from the training set yet remain clinically plausible, enabling interpolation of variance beyond the original samples.