[Paper Review] Robust Collaborative Learning of Patch-level and Image-level Annotations for Diabetic Retinopathy Grading from Fundus Image

Yehui Yang, Fangxin Shang+6 more|arXiv (Cornell University)|Aug 3, 2020

Retinal Imaging and Analysis49 references4 citations

TL;DR

This paper proposes a robust end-to-end framework, CLPI, that collaboratively learns from patch-level lesion annotations and image-level DR severity grades for diabetic retinopathy grading. By using a lesion attention generator trained on sparse patch-level labels and back-propagating grade supervision to refine attention maps, the method achieves state-of-the-art performance and superior robustness to label scarcity and distribution shifts, outperforming both SOTA algorithms and experienced ophthalmologists.

ABSTRACT

Diabetic retinopathy (DR) grading from fundus images has attracted increasing interest in both academic and industrial communities. Most convolutional neural network (CNN) based algorithms treat DR grading as a classification task via image-level annotations. However, these algorithms have not fully explored the valuable information in the DR-related lesions. In this paper, we present a robust framework, which collaboratively utilizes patch-level and image-level annotations, for DR severity grading. By an end-to-end optimization, this framework can bi-directionally exchange the fine-grained lesion and image-level grade information. As a result, it exploits more discriminative features for DR grading. The proposed framework shows better performance than the recent state-of-the-art algorithms and three clinical ophthalmologists with over nine years of experience. By testing on datasets of different distributions (such as label and camera), we prove that our algorithm is robust when facing image quality and distribution variations that commonly exist in real-world practice. We inspect the proposed framework through extensive ablation studies to indicate the effectiveness and necessity of each motivation. The code and some valuable annotations are now publicly available.

Motivation & Objective

• To address the limitation of existing CNN-based DR grading methods that ignore fine-grained lesion information despite its clinical relevance.
• To develop a collaborative learning framework that integrates patch-level lesion annotations and image-level severity grades in an end-to-end manner.
• To improve model robustness against missing annotations and distribution shifts common in real-world fundus imaging.
• To reduce reliance on expensive pixel-level annotations by using only patch-level labels, which are more feasible to collect.
• To achieve performance comparable to senior ophthalmologists while maintaining generalization across diverse datasets.

Proposed method

• A lesion attention generator is trained on a small set of patch-level annotations to produce spatial attention maps for lesion localization.
• The attention maps are generated by expanding a 16×16 probabilistic matrix into a 512×512 attention map through duplication and tiling, preserving spatial structure.
• The grading module uses both the input image and the lesion attention map to predict DR severity, enabling joint optimization.
• Back-propagation from image-level grade supervision fine-tunes the lesion attention generator, enabling bidirectional knowledge transfer.
• The lesion attention generator is pre-trained on image patches and can detect lesions in full images with a single forward pass.
• The framework is trained end-to-end, allowing joint optimization of lesion detection and DR grading without separate two-stage training.

Experimental results

Research questions

• RQ1 Can a collaborative learning framework that jointly uses patch-level lesion and image-level grade annotations improve DR grading performance?
• RQ2 How does the proposed method perform under label scarcity and missing annotations, common in medical imaging?
• RQ3 Does end-to-end optimization of lesion attention and image grading lead to better generalization across diverse data distributions?
• RQ4 Can a lesion attention generator trained on sparse patch-level labels achieve performance comparable to SOTA detectors?
• RQ5 Is the proposed framework robust to variations in image quality and camera types found in real-world clinical settings?

Key findings

• The CLPI framework achieves state-of-the-art performance on DR grading, outperforming recent SOTA algorithms and matching the accuracy of three experienced ophthalmologists with over nine years of clinical experience.
• The method demonstrates robustness to missing lesion annotations: when 50% of lesion labels are discarded, performance degrades significantly less than for SOTA detectors like Faster-RCNN and YOLO-V3.
• The lesion attention generator maintains high performance even with sparse annotations, as it can collect negative patches from No-DR images to mitigate label scarcity.
• Ablation studies confirm that both the lesion attention mechanism and end-to-end collaborative learning are essential for optimal performance.
• The framework generalizes well across datasets with different label distributions and camera types, proving its robustness in real-world deployment scenarios.
• The method requires only patch-level annotations instead of costly pixel-level masks, making it more practical for clinical deployment.