[Paper Review] Data-Driven Color Augmentation Techniques for Deep Skin Image Analysis

Adrián Galdrán, Aitor Álvarez-Gila+8 more|arXiv (Cornell University)|Mar 10, 2017

melanin and skin pigmentation9 references58 citations

TL;DR

The paper introduces a data augmentation method for dermoscopy images that uses Shades of Gray-based illuminant estimation to color-cast white-balanced training images, improving segmentation and classification in ISIC 2017 without external dermoscopy data.

ABSTRACT

Dermoscopic skin images are often obtained with different imaging devices, under varying acquisition conditions. In this work, instead of attempting to perform intensity and color normalization, we propose to leverage computational color constancy techniques to build an artificial data augmentation technique suitable for this kind of images. Specifically, we apply the \emph{shades of gray} color constancy technique to color-normalize the entire training set of images, while retaining the estimated illuminants. We then draw one sample from the distribution of training set illuminants and apply it on the normalized image. We employ this technique for training two deep convolutional neural networks for the tasks of skin lesion segmentation and skin lesion classification, in the context of the ISIC 2017 challenge and without using any external dermatologic image set. Our results on the validation set are promising, and will be supplemented with extended results on the hidden test set when available.

Motivation & Objective

• Motivate robust skin lesion analysis despite varied acquisition conditions and devices.
• Propose a color-constancy–based augmentation to simulate diverse illumination conditions.
• Evaluate augmentation on skin lesion segmentation and classification within ISIC 2017 without external dermatologic data.

Proposed method

• Estimate scene illuminants with Shades of Gray color constancy on training images.
• White-balance images and augment by sampling illuminants from the empirical distribution to create color-casted training samples.
• Apply a Von Kries–like diagonal chromatic adaptation transform for augmentation.
• Apply gamma-correction augmentation by sampling gamma from a truncated normal distribution.
• Use standard geometric (and non-linear) data augmentation to increase data variability.
• Train U-Net for segmentation and 50-layer residual networks for classification with domain-augmented data.

Experimental results

Research questions

• RQ1 Can illuminant-based data augmentation improve generalization of deep models on dermoscopy tasks without external data?
• RQ2 How does the proposed augmentation affect segmentation accuracy and classification performance in ISIC 2017?
• RQ3 Is color constancy-based augmentation robust to variations in illumination and device differences in skin images?

Key findings

• Segmentation accuracy (ISIC 2017 validation): ACC 0.948, Dice 0.846, Jaccard 0.767, Sensitivity 0.865, Specificity 0.980.
• Melanoma classification AUC 0.791, ACC 0.580, AP 0.482, Sensitivity 0.833, Specificity 0.517.
• Seborrheic keratosis classification AUC 0.954, ACC 0.867, AP 0.898, Sensitivity 0.857, Specificity 0.870.
• Average classification metrics: AUC 0.873, ACC 0.723, AP 0.690, Sensitivity 0.845, Specificity 0.694.