[Paper Review] DermGAN: Synthetic Generation of Clinical Skin Images with Pathology

Amirata Ghorbani, Vivek Natarajan+2 more|arXiv (Cornell University)|Nov 20, 2019

AI in cancer detectionComputer Science43 references44 citations

TL;DR

DermGAN adapts Pix2Pix for synthetic clinical skin images with controllable skin condition, size, location, and skin color, validated by GAN metrics and human tests; augmentation with synthetic images improves rare malignant skin conditions in classifier performance.

ABSTRACT

Despite the recent success in applying supervised deep learning to medical imaging tasks, the problem of obtaining large and diverse expert-annotated datasets required for the development of high performant models remains particularly challenging. In this work, we explore the possibility of using Generative Adverserial Networks (GAN) to synthesize clinical images with skin condition. We propose DermGAN, an adaptation of the popular Pix2Pix architecture, to create synthetic images for a pre-specified skin condition while being able to vary its size, location and the underlying skin color. We demonstrate that the generated images are of high fidelity using objective GAN evaluation metrics. In a Human Turing test, we note that the synthetic images are not only visually similar to real images, but also embody the respective skin condition in dermatologists' eyes. Finally, when using the synthetic images as a data augmentation technique for training a skin condition classifier, we observe that the model performs comparably to the baseline model overall while improving on rare but malignant conditions.

Motivation & Objective

• Motivate data scarcity and imbalance in medical skin imaging data, especially for rare malignant conditions.
• Propose a generative framework to synthesize clinical skin images with specified pathology, size, location, and skin color.
• Demonstrate generation quality via objective GAN metrics and human evaluation.
• Show utility of synthetic images as augmentation for skin condition classification, with emphasis on rare classes.

Proposed method

• Adapt Pix2Pix-style conditional GAN to medical skin images with semantic maps encoding skin color, condition presence, and ROI location.
• Modify generator (U-Net with resizing layers to reduce checkerboard artifacts) and a fully-convolutional discriminator.
• Train with four loss terms: image reconstruction loss, condition-specific ROI reconstruction loss, min-max GAN loss, and feature-matching loss.
• Perform hyperparameter tuning via grid search to balance loss components.
• Preprocess dataset by cropping around annotated ROIs to create 256x256 training images and 40000 images for training, 24000 for evaluation.

Experimental results

Research questions

• RQ1 Can a conditional GAN generate realistic clinical skin images from semantic maps specifying pathology, skin color, and ROI?
• RQ2 How do variations in skin color and lesion size affect generated image fidelity and dermatological realism?
• RQ3 Do synthetic images improve skin condition classification, especially for rare malignant classes, when used as data augmentation?
• RQ4 What objective and subjective metrics best reflect realism and clinical relevance of generated images?

Key findings

• DermGAN can generate high-fidelity synthetic images conditioned on skin color, lesion type, and ROI size/location.
• Varying skin color in the semantic map changes background appearance consistently with real-world observations.
• Increasing lesion size in the semantic map yields corresponding visual changes in the generated skin region.
• Objective GAN metrics (FID) improve when using the full DermGAN model versus ablations; baselines show worse FID scores.
• Human Turing tests indicate synthetic images are often indistinguishable from real ones, and dermatologists perceive correct clinical content in synthetic images.
• In data augmentation for skin condition classification, using 20000 synthetic images yielded comparable overall accuracy to baseline but improved F1 for rare malignant classes (Melanoma) and Basal Cell Carcinoma, with a trade-off in Melanocytic nevus.