[Paper Review] Spatially Aware Melanoma Segmentation Using Hybrid Deep Learning Techniques

Mohamed Attia, Mustafa Hossny+2 more|arXiv (Cornell University)|Feb 26, 2017

Cutaneous Melanoma Detection and Management2 references16 citations

TL;DR

This paper proposes a hybrid deep learning model combining convolutional neural networks (CNNs) and recurrent neural networks (RNNs) to improve spatial accuracy in melanoma lesion segmentation. By using RNNs to model spatial dependencies across non-overlapping image patches after CNN encoding, the method achieves superior performance—outperforming FCN and SegNet on the ISBI 2017 challenge with a Jaccard index of 0.93 and Dice coefficient of 0.96—while being robust to artifacts like hair and markers without post-processing.

ABSTRACT

In this paper, we proposed using a hybrid method that utilises deep convolutional and recurrent neural networks for accurate delineation of skin lesion of images supplied with ISBI 2017 lesion segmentation challenge. The proposed method was trained using 1800 images and tested on 150 images from ISBI 2017 challenge.

Motivation & Objective

• To address over-segmentation in skin lesion segmentation caused by coarse max-pooling in fully convolutional networks.
• To improve spatial accuracy in lesion boundary delineation using recurrent modeling of feature dependencies.
• To develop a method that is robust to common imaging artifacts such as hair, markers, and low contrast.
• To eliminate the need for post-processing or contrast enhancement in segmentation outputs.

Proposed method

• The architecture uses a 7-convolutional layer encoder with 2 max-pooling layers to extract deep features from input images.
• Input feature maps are divided into non-overlapping patches, which are flattened and fed into 4-directional RNNs (horizontal and vertical sweeps) to model spatial context.
• Coupled RNNs process data in both forward and reverse directions (e.g., left-to-right and right-to-left) to capture bidirectional spatial dependencies.
• The RNN outputs are concatenated and used as context-aware features for mask reconstruction via a deconvolutional decoder with fractionally strided convolutions.
• The joint training of CNN and RNN components is enabled through computationally efficient strided convolutions.
• Final segmentation masks are reconstructed at full input resolution using learned deconvolutional layers.

Experimental results

Research questions

• RQ1 Can recurrent neural networks effectively model spatial dependencies between feature maps to improve lesion boundary localization in skin lesion segmentation?
• RQ2 Does combining CNNs with RNNs reduce over-segmentation compared to standard FCN and SegNet architectures?
• RQ3 Can the hybrid model maintain high accuracy on low-contrast lesions and lesions with artifacts like hair or markers without post-processing?
• RQ4 How does the proposed method compare quantitatively to state-of-the-art models like FCN and SegNet on the ISBI 2017 benchmark?

Key findings

• The proposed method achieved a Jaccard index of 0.93, significantly outperforming SegNet (0.86) and FCN (0.86) on the ISBI 2017 test set.
• The model demonstrated a Dice coefficient of 0.96, indicating high overlap with ground truth segmentation masks.
• Sensitivity reached 0.954, showing strong detection capability for lesion regions, even in challenging cases.
• The method produced accurate segmentations without requiring contrast enhancement or post-processing, unlike SegNet.
• Visual results confirmed robustness to hair occlusions, marker artifacts, and low-contrast lesions.
• The model maintained high specificity (0.94) and accuracy (0.98), indicating reliable performance across diverse lesion types.