[Paper Review] Selective Synthetic Augmentation with Quality Assurance

Yuan Xue, Jiarong Ye+4 more|arXiv (Cornell University)|Dec 9, 2019

AI in cancer detection34 references4 citations

TL;DR

This paper proposes a selective synthetic augmentation framework that improves medical and natural image classification by filtering high-quality, confidence-verified conditional GAN (cGAN)-generated images before adding them to training data. By combining Monte Carlo dropout for label confidence estimation and feature space distance to real data centroids, the method ensures only high-quality synthetic samples are used, achieving 6.8%, 3.9%, and 1.6% accuracy gains on histopathology, CIFAR10, and SVHN datasets, respectively.

ABSTRACT

Supervised training of an automated medical image analysis system often requires a large amount of expert annotations that are hard to collect. Moreover, the proportions of data available across different classes may be highly imbalanced for rare diseases. To mitigate these issues, we investigate a novel data augmentation pipeline that selectively adds new synthetic images generated by conditional Adversarial Networks (cGANs), rather than extending directly the training set with synthetic images. The selection mechanisms that we introduce to the synthetic augmentation pipeline are motivated by the observation that, although cGAN-generated images can be visually appealing, they are not guaranteed to contain essential features for classification performance improvement. By selecting synthetic images based on the confidence of their assigned labels and their feature similarity to real labeled images, our framework provides quality assurance to synthetic augmentation by ensuring that adding the selected synthetic images to the training set will improve performance. We evaluate our model on a medical histopathology dataset, and two natural image classification benchmarks, CIFAR10 and SVHN. Results on these datasets show significant and consistent improvements in classification performance (with 6.8%, 3.9%, 1.6% higher accuracy, respectively) by leveraging cGAN generated images with selective augmentation.

Motivation & Objective

• To address the challenge of limited, imbalanced medical image datasets requiring high-quality synthetic data for robust model training.
• To overcome the risk of degraded performance when blindly adding visually realistic but low-quality synthetic images to training sets.
• To develop a quality-assured synthetic augmentation pipeline that selectively includes only high-confidence, representative synthetic samples.
• To validate the method’s effectiveness and generalizability across medical and natural image classification tasks.

Proposed method

• A conditional GAN (cGAN) is trained to generate high-fidelity synthetic histopathology patches, with model selection based on a smoothed Fréchet Inception Distance (FID) score.
• A pre-trained ResNet34 with Monte Carlo dropout is used to extract features and compute class-specific centroids in the feature space from real training data.
• Synthetic images are filtered based on predictive entropy from multiple forward passes (Monte Carlo sampling), retaining only those with low entropy (high label confidence).
• Images are further filtered based on their L2 distance to real data class centroids in the feature space, retaining those closest to the true class distribution.
• The final selection of synthetic samples is determined by the augmentation ratio $ r $, ensuring a controlled, quality-driven expansion of the training set.
• The framework is evaluated on a cervical histopathology dataset and standard benchmarks (CIFAR10, SVHN), using ResNet34 as the backbone classifier for fair comparison.

Experimental results

Research questions

• RQ1 Can selective synthetic augmentation using cGAN-generated images improve classification performance on small, imbalanced medical image datasets?
• RQ2 Does filtering synthetic images based on label confidence and feature similarity to real data centroids lead to better model generalization than naive augmentation?
• RQ3 How does the proposed method compare to traditional data augmentation and non-selective synthetic augmentation in terms of accuracy and robustness?
• RQ4 What is the optimal augmentation ratio $ r $ for maximizing performance gains without introducing noisy or misleading samples?
• RQ5 Is the proposed selective augmentation pipeline generalizable across diverse image domains, including medical and natural images?

Key findings

• On the cervical histopathology dataset, the proposed method achieved a 6.8% absolute accuracy gain over the baseline model without any augmentation.
• On CIFAR10 with only 10% of the training data, the selective augmentation method improved accuracy by 3.9 percentage points compared to the baseline.
• The method outperformed non-selective GAN augmentation by 1.6% on SVHN, demonstrating consistent gains across datasets.
• Models trained with non-selective GAN augmentation performed worse than the baseline, confirming that indiscriminate use of synthetic data can degrade performance.
• The ablation study showed that increasing the augmentation ratio $ r $ beyond 0.5 led to performance degradation, highlighting the importance of selective filtering.
• The use of Monte Carlo dropout for uncertainty estimation and feature space distance for distribution alignment proved critical for quality assurance in synthetic data integration.