[Paper Review] Cost-Effective Active Learning for Melanoma Segmentation

Marc Górriz Blanch, Axel Carlier+2 more|arXiv (Cornell University)|Nov 24, 2017

AI in cancer detection8 references72 citations

TL;DR

The paper proposes a Cost-Effective Active Learning framework using Monte Carlo dropout to estimate pixel-wise uncertainty and guide complementary sample selection for melanoma segmentation, achieving 74% Dice after 9 iterations on ISIC 2017 with limited annotations.

ABSTRACT

We propose a novel Active Learning framework capable to train effectively a convolutional neural network for semantic segmentation of medical imaging, with a limited amount of training labeled data. Our contribution is a practical Cost-Effective Active Learning approach using dropout at test time as Monte Carlo sampling to model the pixel-wise uncertainty and to analyze the image information to improve the training performance. The source code of this project is available at https://marc-gorriz.github.io/CEAL-Medical-Image-Segmentation/ .

Motivation & Objective

• Motivate reducing labeling workload in medical image segmentation due to annotation costs.
• Develop a practical CEAL-based framework for CNN semantic segmentation in medical imaging.
• Incorporate pixel-wise uncertainty from Monte Carlo dropout to drive informative sample selection and pseudo-labeling.

Proposed method

• Use Monte Carlo Dropout to approximate the posterior distribution over network weights for pixel-wise uncertainty estimation.
• Compute a pixel-wise uncertainty map by aggregating multiple dropout-enabled predictions across an image.
• Weight the uncertainty map by a distance transform to emphasize contours and refine the final uncertainty score.
• Convert the uncertainty map into a labeling confidence score via summing pixel uncertainties and distance weighting.
• Implement complementary sample selection: label low-confidence samples, pseudo-label high-confidence samples, and include a portion of uncertain samples for iterative refinement.
• Apply to U-Net based semantic segmentation with a CEAL strategy to reduce manual labeling.
• Use ISIC 2017 skin lesion dataset with 600 labeled and 1000 unlabeled samples to simulate active learning.

Experimental results

Research questions

• RQ1 How can pixel-wise uncertainty be effectively leveraged to guide active learning in medical image segmentation?
• RQ2 Can a cost-effective CEAL framework reduce labeling needs while maintaining segmentation performance?
• RQ3 What is the impact of combining uncertain, non-detection, and pseudo-labeled samples on model performance over iterations?
• RQ4 How does distance-weighted uncertainty influence contour accuracy in melanoma segmentation?

Key findings

• Achieved 74% Dice coefficient after 9 active learning iterations on ISIC 2017.
• Demonstrated that combining uncertain, non-detection, random, and pseudo-labeled samples can progressively improve segmentation performance.
• Showed that pixel-wise uncertainty with distance-aware weighting focuses learning on challenging contour regions.
• Indicated that the central region of interest may still limit performance, suggesting room for further improvement in later iterations.