[Paper Review] Skin Lesion Classification Using Hybrid Deep Neural Networks

Amirreza Mahbod, G. Bradley Schaefer+3 more|arXiv (Cornell University)|Feb 27, 2017

Cutaneous Melanoma Detection and Management22 references7 citations

TL;DR

This paper proposes a hybrid deep learning approach for skin lesion classification using pre-trained CNNs (AlexNet, VGG16, ResNet-18) to extract multi-level features, which are then classified using SVMs and fused for improved performance. The method achieves an AUC of 83.83% for melanoma and 97.55% for seborrheic keratosis on the ISIC 2017 validation set, demonstrating the effectiveness of feature and classifier fusion.

ABSTRACT

Skin cancer is one of the major types of cancers with an increasing incidence over the past decades. Accurately diagnosing skin lesions to discriminate between benign and malignant skin lesions is crucial to ensure appropriate patient treatment. While there are many computerised methods for skin lesion classification, convolutional neural networks (CNNs) have been shown to be superior over classical methods. In this work, we propose a fully automatic computerised method for skin lesion classification which employs optimised deep features from a number of well-established CNNs and from different abstraction levels. We use three pre-trained deep models, namely AlexNet, VGG16 and ResNet-18, as deep feature generators. The extracted features then are used to train support vector machine classifiers. In the final stage, the classifier outputs are fused to obtain a classification. Evaluated on the 150 validation images from the ISIC 2017 classification challenge, the proposed method is shown to achieve very good classification performance, yielding an area under receiver operating characteristic curve of 83.83% for melanoma classification and of 97.55% for seborrheic keratosis classification.

Motivation & Objective

• To improve skin lesion classification accuracy by leveraging transfer learning from pre-trained deep neural networks.
• To investigate the impact of combining features from multiple CNN architectures and different abstraction levels on classification performance.
• To develop a fully automatic, end-to-end method for classifying benign nevi, malignant melanoma, and seborrheic keratosis using dermoscopic images.
• To evaluate the effectiveness of SVM-based fusion of multiple classifier outputs for enhanced discrimination between skin lesion types.

Proposed method

• Pre-trained CNNs (AlexNet, VGG16, ResNet-18) are used as feature extractors on dermoscopic images after resizing to input dimensions compatible with each network.
• Deep features are extracted from fully connected (FC) layers and, in some cases, from the last convolutional layer to capture multi-level abstraction.
• Support vector machines (SVMs) are trained on the extracted features for each individual network and lesion class.
• Classifier outputs from individual networks are fused using a weighted averaging strategy to improve overall classification performance.
• The method employs minimal pre-processing, including ImageNet mean subtraction, to maintain generalization across datasets.
• Performance is evaluated on the ISIC 2017 validation set using the area under the ROC curve (AUC).

Fig. 1 : Common artefacts in dermoscopic images from the ISIC challenge. Normal image, dark corner artefact, skin hair artefacts, colour chart artefact, ruler marker artefact, ink marker artefact (left to right, top to bottom).

Experimental results

Research questions

• RQ1 Can combining deep features from multiple pre-trained CNNs improve skin lesion classification beyond single-network approaches?
• RQ2 Does fusing features from different abstraction levels (e.g., FC layers and convolutional layers) lead to better performance than using a single layer?
• RQ3 How does the fusion of multiple SVM classifiers, each trained on features from different networks, compare to individual network performance?
• RQ4 Can transfer learning from ImageNet-pretrained models effectively generalize to the task of dermoscopic skin lesion classification with limited training data?

Key findings

• The fusion of all three networks (AlexNet, VGG16, ResNet-18) using features from all fully connected layers achieved the highest overall AUC of 90.69% on the ISIC 2017 validation set.
• The proposed method achieved an AUC of 83.83% for melanoma classification and 97.55% for seborrheic keratosis, significantly outperforming single-network baselines.
• Using features from all fully connected layers across a single network (e.g., AlexNet) yielded better performance than using only the final FC8 layer, indicating the value of multi-level feature abstraction.
• Among single networks, AlexNet produced the best individual results (AUC 89.73), suggesting that shallower architectures may generalize better on smaller datasets.
• The fusion of AlexNet and ResNet-18 achieved an AUC of 90.29%, demonstrating that combining diverse architectures enhances performance.
• The method achieved strong performance even with minimal pre-processing, indicating robustness and generalization potential across dermoscopic image datasets.

Fig. 3 : ROC curve of the best performing approach.