[Paper Review] Synthetic Medical Images from Dual Generative Adversarial Networks

John Guibas, Tejpal S. Virdi+1 more|arXiv (Cornell University)|Sep 6, 2017

Generative Adversarial Networks and Image Synthesis7 references111 citations

TL;DR

The paper presents a two-stage GAN pipeline that decomposes medical image generation into geometry (stage I) and photorealism (stage II) to create synthetic retinal fundus images, facilitating public data access while preserving privacy.

ABSTRACT

Currently there is strong interest in data-driven approaches to medical image classification. However, medical imaging data is scarce, expensive, and fraught with legal concerns regarding patient privacy. Typical consent forms only allow for patient data to be used in medical journals or education, meaning the majority of medical data is inaccessible for general public research. We propose a novel, two-stage pipeline for generating synthetic medical images from a pair of generative adversarial networks, tested in practice on retinal fundi images. We develop a hierarchical generation process to divide the complex image generation task into two parts: geometry and photorealism. We hope researchers will use our pipeline to bring private medical data into the public domain, sparking growth in imaging tasks that have previously relied on the hand-tuning of models. We have begun this initiative through the development of SynthMed, an online repository for synthetic medical images.

Motivation & Objective

• Motivate and enable public access to medical imaging data by generating realistic synthetic images.
• Reduce reliance on private patient data for training deep learning models in medical imaging.
• Split image generation into a geometry-focused stage and a photorealism-focused stage to improve stability and diversity.

Proposed method

• Stage-I GAN (DCGAN) generates varied segmentation masks representing geometry of images.
• Stage-II GAN (CGAN with image-to-image translation) maps segmentation masks to photorealistic images.
• Training uses DRIVE for segmentation masks and MESSIDOR for photorealistic pairs, plus a comparison with a single GAN model.
• A U-net segmentation network is trained on synthetic data to evaluate usefulness for segmentation tasks.
• Evaluation includes F1 score, KL-divergence between synthetic and real data distributions, and qualitative analysis.

Experimental results

Research questions

• RQ1 Can a dual-GAN pipeline produce synthetic medical images with realistic geometry and photorealism?
• RQ2 Does synthetic data from the pipeline support effective training of medical image segmentation models?
• RQ3 How does the synthetic data distribution compare to real data in terms of KL divergence and F1 performance?
• RQ4 Is the approach generalizable to datasets beyond retinal images (e.g., BU-BIL rat cell images)?

Key findings

• The synthetically trained U-net achieved F1 = 0.8877, close to the DRIVE-trained U-net F1 = 0.8988.
• A KL-divergence of 4.759 for synthetic data shows observable but non-identical distribution differences from real data (vs. 4.212e-4 for real-subset comparison).
• The dual-GAN pipeline improves image quality by separating geometry from photorealism, enabling more stable training and more varied images.
• The pipeline successfully produced synthetic retinal images and segmentation masks, and demonstrated transfer to a different domain (BU-BIL rat cells).
• Qualitative results indicate Stage-II learns geometry-to-photorealism mapping, while Stage-I generates diverse segmentation geometries.