[Paper Review] Semantic Segmentation of Skin Lesions using a Small Data Set

Beril Sırmaçek, Max Kivits|arXiv (Cornell University)|Oct 23, 2019

Cutaneous Melanoma Detection and Management16 references4 citations

TL;DR

This paper proposes a semantic segmentation approach for skin lesions using a small dataset of 41 smartphone-captured images. It demonstrates that fully convolutional networks (FCN16/FCN32) with transfer learning from pre-trained VGG16, combined with data cropping and augmentation, achieve optimal performance even with limited data, outperforming random initialization and simpler architectures in accuracy, IoU, and F1 score.

ABSTRACT

Early detection of melanoma is difficult for the human eye but a crucial step towards reducing its death rate. Computerized detection of these melanoma and other skin lesions is necessary. The central research question in this paper is "How to segment skin lesion images using a neural network with low available data?". This question is divided into three sub questions regarding best performing network structure, training data and training method. First theory associated with these questions is discussed. Literature states that U-net CNN structures have excellent performances on the segmentation task, more training data increases network performance and utilizing transfer learning enables networks to generalize to new data better. To validate these findings in the literature two experiments are conducted. The first experiment trains a network on data sets of different size. The second experiment proposes twelve network structures and trains them on the same data set. The experimental results support the findings in the literature. The FCN16 and FCN32 networks perform best in the accuracy, intersection over union and mean BF1 Score metric. Concluding from these results the skin lesion segmentation network is a fully convolutional structure with a skip architecture and an encoder depth of either one or two. Weights of this network should be initialized using transfer learning from the pre trained VGG16 network. Training data should be cropped to reduce complexity and augmented during training to reduce the likelihood of overfitting.

Motivation & Objective

• To address the challenge of training accurate semantic segmentation models for skin lesions when only limited annotated data is available.
• To investigate the impact of training data size, network architecture, and training methods on segmentation performance in low-data regimes.
• To identify the optimal combination of network structure, weight initialization, and data preprocessing for robust skin lesion segmentation.
• To improve early melanoma detection by enabling precise pixel-level lesion localization using minimal training data.

Proposed method

• Trained fully convolutional networks (FCN16, FCN32, FCN8, and SGN variants) on a small dataset of 41 skin lesion images from a smartphone camera.
• Utilized transfer learning by initializing network weights with features from a pre-trained VGG16 network to improve generalization.
• Applied data augmentation and image cropping to increase effective training data and reduce complexity.
• Employed early stopping regularization to prevent overfitting and reduce training time.
• Compared performance across different network architectures and training strategies using metrics like IoU, accuracy, and F1 score.
• Used two additional public datasets (126 and 10 images) for comparative analysis and generalization evaluation.

Experimental results

Research questions

• RQ1 How does the size of the training data set impact segmentation performance and generalization in skin lesion semantic segmentation?
• RQ2 Which neural network architecture yields the best performance for semantic segmentation of skin lesions with limited data?
• RQ3 How does the choice of training method—specifically transfer learning versus random weight initialization—affect performance and training efficiency?
• RQ4 What preprocessing techniques improve model performance when training data is sparse?

Key findings

• The FCN32 and FCN16 networks achieved the highest performance across all metrics, including accuracy, intersection over union (IoU), and mean F1 score.
• Transfer learning using partially initialized weights from VGG16 significantly improved both specific and general segmentation performance compared to random initialization.
• Networks trained with transfer learning required substantially less training time and showed better generalization, especially on unseen data.
• Cropping the training images improved model generalization, with the SGN3 network trained on cropped data showing superior performance on new samples.
• Increasing the number of convolutional layers per encoding block was more effective than increasing encoder depth for performance gains.
• Skip connections that combined deep, low-level features with high-level, processed features outperformed those combining with more processed features.