[Paper Review] Identifying Melanoma Images using EfficientNet Ensemble: Winning Solution to the SIIM-ISIC Melanoma Classification Challenge

Qishen Ha, Бо Лю+1 more|arXiv (Cornell University)|Oct 11, 2020

AI in cancer detection5 references46 citations

TL;DR

The paper presents a winning ensemble of diverse CNN models (mostly EfficientNets) with image and metadata inputs, using a stable validation scheme and target-aware labeling to achieve top AUC in SIIM-ISIC Melanoma Classification Challenge.

ABSTRACT

We present our winning solution to the SIIM-ISIC Melanoma Classification Challenge. It is an ensemble of convolutions neural network (CNN) models with different backbones and input sizes, most of which are image-only models while a few of them used image-level and patient-level metadata. The keys to our winning are: (1) stable validation scheme (2) good choice of model target (3) carefully tuned pipeline and (4) ensembling with very diverse models. The winning submission scored 0.9600 AUC on cross validation and 0.9490 AUC on private leaderboard.

Motivation & Objective

• Address the challenge of melanoma classification from skin images with limited positive samples by leveraging data from multiple years.
• Develop a stable validation strategy to reliably estimate model performance in imbalanced datasets.
• Explore model architectures with and without metadata to enhance ensemble diversity and performance.
• Demonstrate an ensemble of diverse backbones and input sizes to maximize AUC on cross-validation and private leaderboard.

Proposed method

• Train multiple CNN models with diverse backbones (EfficientNet B3–B7, SE-ResNeXt-101, ResNeSt-101).
• Use a 9-class target derived from mapping 2019 diagnoses to 2020 targets, with MEL as the final prediction for AUC.
• Incorporate 14 metadata features (demographics, anatomy site, image size, n_images) in some models with two FC layers before fusion.
• Apply extensive image augmentations (Albumentations) and cosine annealing with warm restarts for training over 15 epochs.
• Evaluate models with a combined data 5-fold CV (cv_all) spanning 2018–2020 data and track cv_2020 for sanity checks.
• Create a simple average ensemble over 18 models by converting probabilities to uniform scores before averaging.

Experimental results

Research questions

• RQ1 Can an ensemble of diverse CNN architectures achieve higher AUC on SIIM-ISIC melanoma classification than single-model approaches?
• RQ2 Does incorporating metadata alongside image data improve melanoma classification performance and ensemble diversity?
• RQ3 Does a multi-year combined validation strategy provide a more stable and reliable performance estimate for imbalanced medical imaging data?

Key findings

• The 18-model ensemble achieved cv_all 0.9845, cv_2020 0.9600, private_LB 0.9490, and public_LB 0.9442.
• Model variety across backbones, input sizes, and metadata usage contributed to ensemble robustness.
• Image-only models generally outperformed metadata-only models, but metadata models added valuable diversity.
• Using a combined training set from 2018–2020 with a cv_all-based validation yielded more stable AUC estimates than relying on 2020 data alone.