[Paper Review] Lesion-based Contrastive Learning for Diabetic Retinopathy Grading from Fundus Images

Yijin Huang, Li Lin+3 more|arXiv (Cornell University)|Jul 17, 2021

Retinal Imaging and Analysis20 references4 citations

TL;DR

This paper proposes lesion-based contrastive learning (LCL) for diabetic retinopathy (DR) grading, using lesion patches instead of full fundus images to improve feature discriminability. By focusing on pathological regions and optimizing data augmentation, LCL achieves state-of-the-art performance on EyePACS, outperforming SimCLR by 23.34% in linear evaluation and showing superior transferability with limited labeled data.

ABSTRACT

Manually annotating medical images is extremely expensive, especially for large-scale datasets. Self-supervised contrastive learning has been explored to learn feature representations from unlabeled images. However, unlike natural images, the application of contrastive learning to medical images is relatively limited. In this work, we propose a self-supervised framework, namely lesion-based contrastive learning for automated diabetic retinopathy (DR) grading. Instead of taking entire images as the input in the common contrastive learning scheme, lesion patches are employed to encourage the feature extractor to learn representations that are highly discriminative for DR grading. We also investigate different data augmentation operations in defining our contrastive prediction task. Extensive experiments are conducted on the publicly-accessible dataset EyePACS, demonstrating that our proposed framework performs outstandingly on DR grading in terms of both linear evaluation and transfer capacity evaluation.

Motivation & Objective

• To address the challenge of high annotation cost in medical imaging by developing a self-supervised framework for diabetic retinopathy grading.
• To overcome limitations of standard contrastive learning on high-resolution fundus images, where diagnostic features (lesions) are sparse and easily lost during data augmentation.
• To improve feature representation learning by focusing on lesion patches instead of full fundus images, enhancing discriminability for DR grading.
• To investigate the impact of different data augmentation strategies on contrastive learning for fundus images.
• To evaluate the proposed method on large-scale, real-world datasets like EyePACS, demonstrating superior transferability and linear evaluation performance.

Proposed method

• A two-stage framework is used: first, an object detection network is trained on IDRiD to localize lesions in fundus images.
• Lesion patches are extracted from the EyePACS dataset using the trained detector with a confidence threshold (0.7, 0.8, or 0.9).
• Data augmentation operations—cropping, rotation, color distortion, and gray scaling—are applied to lesion patches to generate positive views for contrastive learning.
• A contrastive loss is minimized to maximize similarity between feature embeddings of augmented views of the same lesion patch and minimize similarity between different patches.
• The feature extractor is pre-trained in a self-supervised manner on these lesion patches before fine-tuning on downstream DR grading tasks.
• Performance is evaluated via linear evaluation and transfer capacity evaluation on partial and full EyePACS datasets.

Experimental results

Research questions

• RQ1 Can focusing on lesion patches instead of full fundus images improve self-supervised representation learning for diabetic retinopathy grading?
• RQ2 Which combination of data augmentation operations is most effective for contrastive learning on fundus images with sparse diagnostic features?
• RQ3 How does lesion-based contrastive learning compare to standard contrastive learning (e.g., SimCLR) in terms of linear evaluation and transfer learning performance?
• RQ4 Does the quality of lesion patches (via confidence thresholding) affect the downstream performance of the self-supervised model?
• RQ5 Can the proposed method achieve strong performance with limited labeled data, demonstrating improved transfer capacity?

Key findings

• Lesion-based contrastive learning achieves a quadratic weighted kappa of 66.88% on linear evaluation using the full EyePACS dataset, outperforming SimCLR (128×128) by 23.34 percentage points.
• With only 1% of the training data, the proposed method achieves a kappa of 68.37 in transfer capacity evaluation, surpassing the fully-supervised baseline by 15.01 percentage points.
• Using a higher detection confidence threshold (0.9) improves performance despite fewer training patches, indicating that patch quality is more important than quantity.
• The combination of cropping, rotation, color distortion, and gray scaling yields the best performance (kappa = 62.49) in ablation studies.
• The method shows superior transferability, especially under low-data regimes, confirming its effectiveness in few-shot DR grading scenarios.
• The source code is publicly available at https://github.com/YijinHuang/Lesion-based-Contrastive-Learning.