[Paper Review] An Evaluation of Self-Supervised Pre-Training for Skin-Lesion Analysis

Levy Chaves, Alceu Bissoto+2 more|arXiv (Cornell University)|Jun 17, 2021

Cutaneous Melanoma Detection and Management4 citations

TL;DR

This paper evaluates self-supervised pre-training methods—BYOL, InfoMin, MoCo, SimCLR, and SwAV—for skin-lesion classification, comparing them to a supervised ImageNet baseline across five datasets with in- and out-of-distribution samples. Self-supervision significantly improves accuracy and reduces performance variability, especially in low-data regimes (<150 samples), demonstrating superior stability and robustness for medical image analysis.

ABSTRACT

Self-supervised pre-training appears as an advantageous alternative to supervised pre-trained for transfer learning. By synthesizing annotations on pretext tasks, self-supervision allows to pre-train models on large amounts of pseudo-labels before fine-tuning them on the target task. In this work, we assess self-supervision for the diagnosis of skin lesions, comparing three self-supervised pipelines to a challenging supervised baseline, on five test datasets comprising in- and out-of-distribution samples. Our results show that self-supervision is competitive both in improving accuracies and in reducing the variability of outcomes. Self-supervision proves particularly useful for low training data scenarios ($&lt;1\,500$ and $&lt;150$ samples), where its ability to stabilize the outcomes is essential to provide sound results.

Motivation & Objective

• To assess the effectiveness of self-supervised pre-training for skin-lesion classification in comparison to supervised pre-training.
• To evaluate performance across diverse datasets, including in-distribution and out-of-distribution test sets.
• To investigate the impact of self-supervised learning in low-data training regimes (as few as 148 samples).
• To analyze whether self-supervised models learn more robust and generalizable representations than supervised baselines.
• To explore the utility of contrastive learning as an intermediate pre-training step before fine-tuning.

Proposed method

• Employed five self-supervised pre-training methods: BYOL, InfoMin, MoCo, SimCLR, and SwAV, all using ResNet-50 as the backbone.
• Pre-trained models on ImageNet using pretext tasks such as contrastive learning, rotation prediction, and instance discrimination.
• Applied a two-stage transfer learning protocol: pre-train on self-supervised pretext tasks, then fine-tune on target skin-lesion datasets.
• Used data augmentation strategies (crop, color jitter, Gaussian blur) to generate positive pairs for contrastive learning.
• Evaluated models on five public skin-lesion datasets (ISIC 2017, 2018, 2019, HAM10000, Derm7pt), including in- and out-of-distribution splits.
• Applied Score-CAM for gradient-free visualization of attention maps to interpret model decisions and compare feature activation patterns.

Experimental results

Research questions

• RQ1 How do self-supervised pre-training methods compare to a supervised ImageNet baseline in skin-lesion classification accuracy across diverse datasets?
• RQ2 Does self-supervised pre-training improve performance and reduce variance in low-data training scenarios (e.g., <150 samples)?
• RQ3 How do self-supervised models perform on out-of-distribution test sets compared to the supervised baseline?
• RQ4 What differences exist in feature activation patterns and attention maps between self-supervised and supervised models?
• RQ5 Can contrastive learning-based self-supervised pre-training serve as a stable and effective alternative to supervised pre-training in medical imaging?

Key findings

• Self-supervised pre-training consistently outperformed the supervised ImageNet baseline in terms of mean accuracy across all five test datasets.
• The performance variance across repeated runs was significantly lower for self-supervised models, especially in low-data regimes.
• In the 1% training data regime (148 samples), self-supervised models reduced performance variability and improved mean accuracy compared to the supervised baseline.
• BYOL, SimCLR, and SwAV emerged as the top-performing self-supervised methods, with higher accuracy and more stable predictions than the supervised baseline.
• Score-CAM visualization revealed that self-supervised models focused on more consistent and biologically plausible lesion regions, especially in low-data settings.
• The double pre-training pipeline—self-supervised pre-training followed by fine-tuning—proved particularly effective in stabilizing results under data scarcity.