[Paper Review] Weakly-supervised localization of diabetic retinopathy lesions in retinal fundus images

Waleed M. Gondal, Jan M. Köhler+3 more|arXiv (Cornell University)|Jun 29, 2017

Retinal Imaging and Analysis28 references9 citations

TL;DR

This paper proposes a weakly-supervised deep learning method for localizing diabetic retinopathy lesions in retinal fundus images using only image-level labels. By adapting a top-performing CNN architecture with global average pooling and class activation mapping, the model achieves high sensitivity (97.2% for hemorrhages) and low false positives (1.5–2.25 FPs/image), outperforming fully supervised methods on lesion-level detection while maintaining an AUC of 0.954 for image-level classification.

ABSTRACT

Convolutional neural networks (CNNs) show impressive performance for image classification and detection, extending heavily to the medical image domain. Nevertheless, medical experts are sceptical in these predictions as the nonlinear multilayer structure resulting in a classification outcome is not directly graspable. Recently, approaches have been shown which help the user to understand the discriminative regions within an image which are decisive for the CNN to conclude to a certain class. Although these approaches could help to build trust in the CNNs predictions, they are only slightly shown to work with medical image data which often poses a challenge as the decision for a class relies on different lesion areas scattered around the entire image. Using the DiaretDB1 dataset, we show that on retina images different lesion areas fundamental for diabetic retinopathy are detected on an image level with high accuracy, comparable or exceeding supervised methods. On lesion level, we achieve few false positives with high sensitivity, though, the network is solely trained on image-level labels which do not include information about existing lesions. Classifying between diseased and healthy images, we achieve an AUC of 0.954 on the DiaretDB1.

Motivation & Objective

• To develop a weakly-supervised method that localizes diabetic retinopathy lesions in retinal fundus images without requiring lesion-level annotations.
• To improve interpretability of deep learning models in medical imaging by highlighting discriminative regions responsible for classification decisions.
• To achieve high performance in lesion detection comparable to fully supervised methods despite training only on image-level labels.
• To address the challenge of scattered, small lesion regions (e.g., microaneurysms, hemorrhages) in retinal images that are difficult to localize with standard CNNs.

Proposed method

• Adapts a state-of-the-art supervised CNN architecture for diabetic retinopathy classification to enable weakly-supervised object localization.
• Uses global average pooling (GAP) to establish a linear relationship between the final convolutional layer and softmax classification output.
• Applies class activation mapping (CAM) to generate heatmaps that highlight discriminative regions responsible for the predicted class.
• Trains the network using only image-level labels (e.g., 'diseased' or 'healthy') without bounding boxes or pixel-level annotations.
• Employs a single forward pass to generate localization maps, enabling fast inference and interpretability.
• Uses overlap-based evaluation (50% and one-pixel thresholds) to assess lesion-level detection performance.

Experimental results

Research questions

• RQ1 Can a weakly-supervised deep learning model localize diabetic retinopathy lesions in retinal fundus images using only image-level labels?
• RQ2 How does the performance of weakly-supervised localization compare to fully supervised methods in detecting scattered, small lesions?
• RQ3 To what extent do the predicted activation maps align with clinically relevant lesion regions?
• RQ4 Can the model maintain high classification accuracy while achieving precise lesion localization without explicit localization annotations?

Key findings

• The model achieves an AUC of 0.954 for image-level classification of diabetic retinopathy on the DiaretDB1 dataset.
• For hemorrhage detection, the model reaches 97.2% sensitivity with only 2.25 false positives per image under the 50% overlap criterion.
• With one-pixel overlap threshold, sensitivity increases to 91% for hemorrhages and 87% for hard exudates, while maintaining 1.5 false positives per image.
• The method outperforms prior weakly-supervised approaches in reducing false positives, achieving 1.45 FPs/image for soft exudates compared to 10 in Quellec et al. [25].
• The model shows limited performance in detecting red small dots (RSD), with only 52% sensitivity under one-pixel overlap, likely due to their sub-pixel size and coarse heatmap resolution.
• Despite no lesion-level supervision, the localization maps strongly correlate with clinically relevant lesion areas, demonstrating robust interpretability.