[Paper Review] Skin cancer detection based on deep learning and entropy to detect outlier samples

André G. C. Pacheco, Abder-Rahman Ali+1 more|arXiv (Cornell University)|Sep 10, 2019

Anomaly Detection Techniques and ApplicationsComputer Science20 references50 citations

TL;DR

The paper builds an ensemble of 13 CNNs to classify eight skin lesion types and detects an outlier UNK class; it also uses entropy-based outlier detection and meta-data to boost performance, achieving competitive balanced accuracy and high AUC.

ABSTRACT

We describe our methods that achieved the 3rd and 4th places in tasks 1 and 2, respectively, at ISIC challenge 2019. The goal of this challenge is to provide the diagnostic for skin cancer using images and meta-data. There are nine classes in the dataset, nonetheless, one of them is an outlier and is not present on it. To tackle the challenge, we apply an ensemble of classifiers, which has 13 convolutional neural networks (CNN), we develop two approaches to handle the outlier class and we propose a straightforward method to use the meta-data along with the images. Throughout this report, we detail each methodology and parameters to make it easy to replicate our work. The results obtained are in accordance with the previous challenges and the approaches to detect the outlier class and to address the meta-data seem to be work properly.

Motivation & Objective

• Motivate automated skin cancer diagnosis from dermoscopic images using ISIC 2019 data.
• Develop an ensemble of CNN models to classify eight known lesion classes.
• Propose methods to detect an outlier class (UNK) and compare approaches.
• Incorporate meta-data (age, sex, region) to improve classification where available.

Proposed method

• Train 13 CNN architectures (SENet, PNASNet, InceptionV4, ResNet-50/101/152, DenseNet-121/169/201, MobileNetV2, GoogleNet, VGG-16/19) pretrained on ImageNet and fine-tuned for 150 epochs with Adam.
• Use class-weighted cross-entropy to address dataset imbalance; explore upsampling but prefer weighted loss.
• Evaluate two ensembles: Ensemble 1 (all 13 models) and Ensemble 2 (top 3 models by balanced accuracy); combine predictions by averaging probabilities.
• Develop two outlier detection approaches: (i) hierarchical classifier to separate skin images (baseline) and (ii) entropy-based selection using per-class entropy thresholds and cosine similarity with hit/miss distributions.
• In Task 2, augment CNN outputs with meta-data via histogram-based probabilities conditioned on age, sex, and region, adjusting top predictions accordingly.

Experimental results

Research questions

• RQ1 Can an ensemble of state-of-the-art CNNs achieve competitive eight-class skin lesion classification on ISIC 2019?
• RQ2 How to detect the UNK outlier class effectively within the ISIC 2019 test set?
• RQ3 Does incorporating meta-data (age, sex, region) improve classification performance for skin lesion diagnosis?
• RQ4 What is the comparative impact of entropy-based outlier detection versus a hierarchical skin-image detector?

Key findings

• Ensemble 2 achieves the highest balanced accuracy among evaluated models (0.897) and accuracy (0.910) with AUC 0.989.
• Ensemble 2 with meta-data slightly improves balanced accuracy to 0.901 and accuracy to 0.910 (AUC 0.987).
• Meta-data integration provides a modest but beneficial boost to performance compared to not using meta-data.
• Entropy-based outlier detection identifies 944 outliers (11.45%) for Ensemble 1 and 579 outliers (7.02%) for Ensemble 2, outperforming the hierarchical approach in detecting unknown samples.
• The eight-class classification performance is competitive with prior ISIC challenges, with SENet being among the strongest single models and ensembles performing best.
• The proposed entropy-based outlier method offers broader outlier detection at the cost of including more samples as unknown, suggesting room for refinement.