[Paper Review] Performance of GAN-based augmentation for deep learning COVID-19 image classification

Oleksandr Fedoruk, K. Klimaszewski+2 more|arXiv (Cornell University)|Apr 18, 2023

COVID-19 diagnosis using AI51 references4 citations

TL;DR

This study evaluates GAN-based data augmentation using StyleGAN2-ADA to improve deep learning classification of COVID-19 chest X-rays across four classes (COVID-19, viral pneumonia, bacterial pneumonia, normal). Despite generating high-quality synthetic images, GAN augmentation underperformed classical data augmentation and simple class balancing, with EfficientNet-B0 achieving 90.2% accuracy on balanced data without augmentation—indicating that for this dataset, additional GAN-generated data provided no benefit and may even degrade performance.

ABSTRACT

The biggest challenge in the application of deep learning to the medical domain is the availability of training data. Data augmentation is a typical methodology used in machine learning when confronted with a limited data set. In a classical approach image transformations i.e. rotations, cropping and brightness changes are used. In this work, a StyleGAN2-ADA model of Generative Adversarial Networks is trained on the limited COVID-19 chest X-ray image set. After assessing the quality of generated images they are used to increase the training data set improving its balance between classes. We consider the multi-class classification problem of chest X-ray images including the COVID-19 positive class that hasn't been yet thoroughly explored in the literature. Results of transfer learning-based classification of COVID-19 chest X-ray images are presented. The performance of several deep convolutional neural network models is compared. The impact on the detection performance of classical image augmentations i.e. rotations, cropping, and brightness changes are studied. Furthermore, classical image augmentation is compared with GAN-based augmentation. The most accurate model is an EfficientNet-B0 with an accuracy of 90.2 percent, trained on a dataset with a simple class balancing. The GAN augmentation approach is found to be subpar to classical methods for the considered dataset.

Motivation & Objective

• To address the challenge of limited, imbalanced medical imaging data in deep learning for COVID-19 classification.
• To investigate whether GAN-based data augmentation improves performance in multi-class COVID-19 X-ray classification beyond classical augmentation.
• To validate the quality and representativeness of GAN-generated synthetic X-ray images using multiple image similarity metrics.
• To compare the performance of multiple deep learning models (EfficientNet-B0, Inception-v3) under various data augmentation strategies.
• To assess model interpretability via saliency maps and evaluate whether models learn biologically plausible features or 'shortcuts'.

Proposed method

• Trained a StyleGAN2-ADA model on a limited, imbalanced dataset of 4-class chest X-rays (COVID-19, viral, bacterial pneumonia, normal) to generate synthetic images.
• Evaluated GAN-generated image quality using FID, RMSE, SRE, and SSIM, comparing intra-class and inter-class similarity distributions.
• Applied classical data augmentation (rotations, cropping, brightness changes) and compared it with GAN-based augmentation across multiple training scenarios.
• Fine-tuned and evaluated two transfer learning models: EfficientNet-B0 and Inception-v3 on balanced and augmented datasets.
• Used Stuart-Maxwell test to assess statistical significance of performance differences across augmentation strategies.
• Analyzed model attention using saliency maps to detect potential shortcut learning and assess localization of attention to lung regions.

Experimental results

Research questions

• RQ1 Does GAN-based data augmentation improve the performance of deep learning models in multi-class COVID-19 X-ray classification compared to classical data augmentation and simple class balancing?
• RQ2 How well do GAN-generated synthetic X-ray images match the distribution and quality of real images, as measured by FID and similarity metrics?
• RQ3 Is the performance degradation observed with GAN augmentation due to poor-quality synthetic images or inherent limitations in the data distribution?
• RQ4 Do deep learning models trained with GAN-augmented data learn meaningful attention patterns in the lungs, or do they rely on spurious correlations?
• RQ5 Can saliency maps confirm that models focus on biologically relevant regions (e.g., lungs) rather than artifacts or anatomical structures like the diaphragm?

Key findings

• The most accurate model was EfficientNet-B0 trained on a balanced dataset without any augmentation, achieving 90.2% accuracy.
• GAN-based augmentation consistently underperformed both classical augmentation and simple class balancing across all evaluated models and metrics.
• Classical data augmentation improved performance for Inception-v3 but provided no additional benefit for EfficientNet-B0, which already achieved high accuracy with balanced data alone.
• Image quality metrics (FID, RMSE, SRE, SSIM) confirmed that GAN-generated images were highly similar to real images, indicating high-quality synthesis.
• Saliency map analysis showed that models focused primarily on lung regions, though some attention was also drawn to non-diagnostic areas like the diaphragm, suggesting minimal shortcut learning.
• The study found that for this specific dataset, the complexity of GAN augmentation did not translate into performance gains, indicating that for sufficiently balanced datasets, additional synthetic data may not be beneficial.