[Paper Review] Deep-Learning Ensembles for Skin-Lesion Segmentation, Analysis, Classification: RECOD Titans at ISIC Challenge 2018

Alceu Bissoto, Fábio Perez+4 more|arXiv (Cornell University)|Aug 25, 2018

Cutaneous Melanoma Detection and Management20 citations

TL;DR

This paper presents a deep-learning ensemble approach using transfer learning, synthetic lesion generation, and advanced data augmentation to improve skin-lesion segmentation, attribute detection, and melanoma classification in the ISIC Challenge 2018. The method achieved a normalized multi-class accuracy of 0.803 on the test set, ranking 9th out of 141 submissions in the classification task.

ABSTRACT

This extended abstract describes the participation of RECOD Titans in parts 1 to 3 of the ISIC Challenge 2018 "Skin Lesion Analysis Towards Melanoma Detection" (MICCAI 2018). Although our team has a long experience with melanoma classification and moderate experience with lesion segmentation, the ISIC Challenge 2018 was the very first time we worked on lesion attribute detection. For each task we submitted 3 different ensemble approaches, varying combinations of models and datasets. Our best results on the official testing set, regarding the official metric of each task, were: 0.728 (segmentation), 0.344 (attribute detection) and 0.803 (classification). Those submissions reached, respectively, the 56th, 14th and 9th places.

Motivation & Objective

• To address the challenge of limited and imbalanced dermatoscopic datasets in skin-lesion analysis by leveraging external data sources and synthetic data generation.
• To improve performance in skin-lesion segmentation, attribute detection, and melanoma classification using deep learning ensembles with transfer learning.
• To investigate the impact of data augmentation, model ensembling, and synthetic lesion generation on model generalization and robustness.
• To develop a scalable and reproducible pipeline for skin-lesion analysis that balances complexity with performance gains.
• To contribute a new method for generating realistic synthetic dermatoscopic images using GANs to alleviate data scarcity in rare lesion types.

Proposed method

• Constructed a comprehensive dataset of 30,726 dermoscopic images by combining ISIC 2018, ISIC Archive, and additional sources, including 631 extra images for underrepresented classes.
• Applied class-weighted cross-entropy loss with frequency-based class weights to mitigate class imbalance during model training.
• Employed extensive online data augmentation including random crops, flips, rotations, shears, scaling, and color jittering (saturation, brightness, contrast, hue).
• Used test-time augmentation with 128 replicas per image, averaging predictions to improve robustness and generalization.
• Implemented a three-part submission strategy: (1) XGBoost ensemble of 43 models, (2) average of 8 top-performing models with synthetic data, and (3) average of 15 models trained only on challenge data.
• Utilized a two-stage training pipeline: initial model selection on holdout sets, followed by meta-learning using validation metrics to refine final ensembles.

Experimental results

Research questions

• RQ1 How does the inclusion of external and synthetic data improve performance in skin-lesion segmentation and classification under severe class imbalance?
• RQ2 What is the impact of advanced data augmentation and test-time augmentation on model generalization and robustness?
• RQ3 Can ensemble learning with diverse deep learning architectures outperform individual models in multi-task skin-lesion analysis?
• RQ4 To what extent can GAN-generated synthetic lesions improve training for rare dermatological conditions?
• RQ5 How do different model architectures and training strategies affect performance across segmentation, attribute detection, and classification tasks?

Key findings

• The ensemble model using 43 deep learning models achieved a normalized multi-class accuracy of 0.732, ranking 32nd out of 141 submissions in the classification task.
• The average of 8 top-performing models trained with synthetic data achieved 0.725 normalized accuracy, ranking 39th, demonstrating the benefit of data augmentation despite limited gains.
• The average of 15 models trained exclusively on challenge data achieved the highest performance with 0.803 normalized accuracy, securing 9th place overall.
• The use of synthetic lesions enabled improved representation of rare classes such as Basal Cell Carcinoma (BCC), Dermatofibroma (DF), and Vascular lesions (VASC), increasing their proportion in the training set.
• Test-time augmentation with 128 replicas significantly improved model robustness, particularly in the presence of noisy or ambiguous annotations.
• The team’s approach demonstrated that simplicity and performance gains must be balanced, as added complexity did not always yield proportional improvements in metrics.