[Paper Review] RECOD Titans at ISIC Challenge 2017

Afonso Menegola, Julia Tavares+4 more|arXiv (Cornell University)|Mar 14, 2017

Cutaneous Melanoma Detection and ManagementMedicine9 references86 citations

TL;DR

The paper documents RECOD Titans’ ISIC 2017 participation: Part 1 uses a U-net-based segmentation baseline with transfer learning; Part 3 tackles melanoma classification with deep models, extensive data gathering, augmentation, and model ensembling, achieving an official validation AUC of 0.793.

ABSTRACT

This extended abstract describes the participation of RECOD Titans in parts 1 and 3 of the ISIC Challenge 2017 "Skin Lesion Analysis Towards Melanoma Detection" (ISBI 2017). Although our team has a long experience with melanoma classification, the ISIC Challenge 2017 was the very first time we worked on skin-lesion segmentation. For part 1 (segmentation), our final submission used four of our models: two trained with all 2000 samples, without a validation split, for 250 and for 500 epochs respectively; and other two trained and validated with two different 1600/400 splits, for 220 epochs. Those four models, individually, achieved between 0.780 and 0.783 official validation scores. Our final submission averaged the output of those four models achieved a score of 0.793. For part 3 (classification), the submitted test run as well as our last official validation run were the result from a meta-model that assembled seven base deep-learning models: three based on Inception-V4 trained on our largest dataset; three based on Inception trained on our smallest dataset; and one based on ResNet-101 trained on our smaller dataset. The results of those component models were stacked in a meta-learning layer based on an SVM trained on the validation set of our largest dataset.

Motivation & Objective

• Motivate melanoma/skin-lesion classification via deep learning in data-scarce medical settings.
• Describe Part 1 (segmentation) and Part 3 (classification) contributions to ISIC Challenge 2017.
• Show how transfer learning, data augmentation, and ensembling improve performance.

Proposed method

• Part 1 uses a U-net-like segmentation network as baseline, with public datasets and online augmentation.
• Transfers from ImageNet-pretrained networks (VGG-16 based), Dice loss, Adam optimizer, and ground-truth mask preprocessing.
• Experimentation with multiple network architectures (VGG-based, U-net variants, batch normalization) and model ensembles to maximize official validation scores.

Experimental results

Research questions

• RQ1 How does transfer learning from large-scale image datasets impact skin-lesion segmentation and classification performance?
• RQ2 What data augmentation and normalization strategies best improve segmentation accuracy and classification AUC in melanoma tasks?
• RQ3 Can ensemble/meta-learning approaches outperform individual models for ISIC 2017 tasks?
• RQ4 What is the impact of dataset composition and deduplication on internal vs. official validation performance?

Key findings

• Final Part 1 submission achieved official validation scores between 0.780 and 0.783 for individual models, with an averaged ensemble score of 0.793.
• Deeper models combined with larger datasets and augmentation yield the best performance; per-image normalization provided gains in some models.
• Stacking models with a meta-learning SVM generally improved results over single-model predictions.
• Using semi-deploy and deploy datasets, plus an aggressive meta-learning ensemble, yielded the best official validation AUC.
• Several explored ideas (e.g., larger image resolution, complex weighting, curriculum learning) did not improve performance under time constraints.