[Paper Review] Fully Convolutional Neural Network for Semantic Segmentation of Anatomical Structure and Pathologies in Colour Fundus Images Associated with Diabetic Retinopathy

Oindrila Saha, Rachana Sathish+1 more|arXiv (Cornell University)|Feb 7, 2019

Retinal Imaging and AnalysisMedicine7 references3 citations

TL;DR

This paper proposes a fully convolutional neural network based on an encoder-decoder architecture inspired by VGG16 and SegNet for end-to-end semantic segmentation of retinal structures and diabetic retinopathy lesions in color fundus images. The model achieves a Jaccard index of 0.8572 for optic disk segmentation and improves lesion detection by including the optic disk as a separate class, with optimized thresholds selected via sensitivity-PPV trade-off analysis.

ABSTRACT

Diabetic retinopathy (DR) is the most common form of diabetic eye disease. Retinopathy can affect all diabetic patients and becomes particularly dangerous, increasing the risk of blindness, if it is left untreated. The success rate of its curability solemnly depends on diagnosis at an early stage. The development of automated computer aided disease diagnosis tools could help in faster detection of symptoms with a wider reach and reasonable cost. This paper proposes a method for the automated segmentation of retinal lesions and optic disk in fundus images using a deep fully convolutional neural network for semantic segmentation. This trainable segmentation pipeline consists of an encoder network, a corresponding decoder network followed by pixel-wise classification to segment microaneurysms, hemorrhages, hard exudates, soft exudates, optic disk from background. The network was trained using Binary cross entropy criterion with Sigmoid as the last layer, while during an additional SoftMax layer was used for boosting response of single class. The performance of the proposed method is evaluated using sensitivity, positive prediction value (PPV) and accuracy as the metrices. Further, the position of the Optic disk is localised using the segmented output map.

Motivation & Objective

• To develop an end-to-end trainable deep learning framework for semantic pixel-wise segmentation of anatomical structures and pathologies in color fundus images associated with diabetic retinopathy.
• To improve early detection of diabetic retinopathy by accurately segmenting microaneurysms, hemorrhages, hard and soft exudates, and the optic disk.
• To address challenges in distinguishing the optic disk from hard exudates by including it as a separate class in the segmentation task.
• To localize the optic disk via centroid detection from the segmented output, enabling anatomical reference for disease assessment.
• To eliminate manual feature engineering and provide a unified solution for multiple subtasks in DR screening.

Proposed method

• The method employs an encoder-decoder architecture with 13 convolutional layers in the encoder, topologically identical to the first 13 layers of VGG16, followed by a corresponding decoder network.
• Each encoder block applies convolution, batch normalization, ReLU activation, and 2×2 max-pooling with stride 2 to downsample feature maps.
• The decoder upsamples the feature maps to match the original input resolution using learned deconvolutional layers and skip connections to preserve spatial details.
• The final layer uses a Sigmoid activation for binary classification per class, followed by a Softmax layer during inference to normalize class probabilities across channels.
• The network is trained using Binary Cross-Entropy loss with Adam optimizer (initial learning rate 10⁻³, β₁ = 0.9), and early stopping is applied based on validation loss.
• Model ensembling is used by saving the best-performing model at different epochs for each class to mitigate overfitting and improve generalization.

Experimental results

Research questions

• RQ1 Can a fully convolutional neural network effectively perform joint semantic segmentation of multiple retinal pathologies and anatomical structures in color fundus images?
• RQ2 Does including the optic disk as a separate class improve the model's ability to distinguish it from hard exudates, especially given their similar brightness levels?
• RQ3 How does the use of global context through an encoder-decoder architecture compare to patch-based training in terms of lesion segmentation accuracy?
• RQ4 What threshold values optimize the trade-off between sensitivity and positive predictive value (PPV) for each lesion type?
• RQ5 To what extent can the segmented output enable accurate localization of the optic disk without using ground-truth locations during training?

Key findings

• The model achieved a Jaccard index (IoU) of 0.8572 for optic disk segmentation, indicating high overlap with ground truth.
• The area under the PPV vs. sensitivity curve was 0.0059 for microaneurysms, 0.5498 for hard exudates, 0.0829 for hemorrhages, and 0.1823 for soft exudates, reflecting varying detection performance across lesion types.
• The mean Euclidean distance between predicted and ground-truth optic disk centroids was 65.93 pixels across 413 test images, demonstrating reasonable localization accuracy.
• Including the optic disk as a separate class significantly improved segmentation performance compared to a 5-class setup, particularly in reducing confusion between exudates and the optic disk.
• The use of a Softmax layer during inference improved mask quality, bringing predicted outputs closer to the ground truth than without Softmax.
• The model outperformed patch-based training, which failed to distinguish the optic disk from exudates due to lack of global context.