[Paper Review] COVID-19 CT Image Synthesis with a Conditional Generative Adversarial Network

Yifan Jiang, Han Chen+2 more|arXiv (Cornell University)|Jul 29, 2020

COVID-19 diagnosis using AI51 references19 citations

TL;DR

This paper proposes a conditional generative adversarial network (cGAN) with a global-local generator and multi-resolution discriminator to synthesize high-fidelity COVID-19 CT images for medical AI training. The method uses dynamic element-wise sum (DESUM) and dynamic feature matching (DFM) mechanisms to balance global structure and local lesion details, achieving state-of-the-art performance with FID of 0.0327 and Dice score of 89.19% in segmentation tasks.

ABSTRACT

Coronavirus disease 2019 (COVID-19) is an ongoing global pandemic that has spread rapidly since December 2019. Real-time reverse transcription polymerase chain reaction (rRT-PCR) and chest computed tomography (CT) imaging both play an important role in COVID-19 diagnosis. Chest CT imaging offers the benefits of quick reporting, a low cost, and high sensitivity for the detection of pulmonary infection. Recently, deep-learning-based computer vision methods have demonstrated great promise for use in medical imaging applications, including X-rays, magnetic resonance imaging, and CT imaging. However, training a deep-learning model requires large volumes of data, and medical staff faces a high risk when collecting COVID-19 CT data due to the high infectivity of the disease. Another issue is the lack of experts available for data labeling. In order to meet the data requirements for COVID-19 CT imaging, we propose a CT image synthesis approach based on a conditional generative adversarial network that can effectively generate high-quality and realistic COVID-19 CT images for use in deep-learning-based medical imaging tasks. Experimental results show that the proposed method outperforms other state-of-the-art image synthesis methods with the generated COVID-19 CT images and indicates promising for various machine learning applications including semantic segmentation and classification.

Motivation & Objective

• Address the scarcity of labeled COVID-19 CT scans due to high infection risk and expert labeling burden.
• Overcome data scarcity in deep learning for medical imaging by generating realistic synthetic CT images.
• Improve generalization and performance of downstream AI tasks like segmentation and classification using synthetic data.
• Develop a robust generative model that preserves both global anatomy and fine-grained pathological features (e.g., ground-glass opacity, consolidation).
• Stabilize training and enhance image quality through novel architectural components like DESUM and DFM.

Proposed method

• Introduces a dual generator structure: a global generator for anatomical structure and a local generator for lesion details, with dynamic element-wise sum (DESUM) to balance their outputs.
• Employs a multi-resolution discriminator with two sub-discriminators: one processing full-resolution images and another processing half-resolution images to capture diverse spatial details.
• Applies dynamic feature matching (DFM) in the discriminator to adaptively weight loss contributions from different resolutions, enhancing training stability and feature representation.
• Uses a conditional cGAN framework where the input condition is a semantic segmentation map of the lung, including regions for lung, ground-glass opacity, and consolidation.
• Leverages a loss function combining adversarial loss, perceptual loss, and L1 reconstruction loss to ensure realistic image generation and high fidelity to real data.
• Optimizes hyperparameters such as the number of generators and discriminators (G=3, D=3) to balance performance, efficiency, and training stability.

Experimental results

Research questions

• RQ1 Can a conditional GAN with dual generators and multi-resolution discriminators generate realistic COVID-19 CT images that preserve both global anatomy and local pathological features?
• RQ2 Does the dynamic element-wise sum (DESUM) mechanism improve image quality by adaptively balancing global and local features during image synthesis?
• RQ3 Can dynamic feature matching (DFM) in the discriminator enhance training stability and improve the realism of generated CT images?
• RQ4 How does the proposed method compare to state-of-the-art image synthesis models in terms of image quality and downstream segmentation performance?
• RQ5 What is the optimal configuration of generator and discriminator depth (G and D) for synthetic CT image generation in terms of FID, PSNR, and Dice score?

Key findings

• The proposed method achieves a Fréchet Inception Distance (FID) of 0.0327, outperforming all compared state-of-the-art methods in image quality.
• The model achieves a peak signal-to-noise ratio (PSNR) of 26.89, indicating high reconstruction fidelity between real and synthetic images.
• In semantic segmentation, the synthetic images yield a Dice score of 89.19% ± 0.24, demonstrating strong compatibility with downstream medical AI tasks.
• Ablation studies show that removing DESUM reduces the Dice score to 84.60% ± 0.39, confirming its critical role in balancing feature fusion.
• The dynamic feature matching (DFM) mechanism improves performance over fixed-weight baselines, with FID dropping from 0.0381 (w/o DFM) to 0.0327 (with DFM).
• Optimal configuration is found to be G=3 generators and D=3 discriminators, as models with fewer or more components show degraded performance (e.g., FID=0.0604 for G=1).