[Paper Review] Detecting Severity of Diabetic Retinopathy from Fundus Images: A Transformer Network-based Review

Tejas Mohan Karkera, Chandranath Adak+2 more|arXiv (Cornell University)|Jan 3, 2023

Retinal Imaging and AnalysisMedicine58 references9 citations

TL;DR

The paper ensembles four image transformer models (ViT, BEiT, CaiT, DeiT) to automatically grade diabetic retinopathy severity from fundus images, achieving state-of-the-art accuracy on APTOS-2019.

ABSTRACT

Diabetic Retinopathy (DR) is considered one of the significant concerns worldwide, primarily due to its impact on causing vision loss among most people with diabetes. The severity of DR is typically comprehended manually by ophthalmologists from fundus photography-based retina images. This paper deals with an automated understanding of the severity stages of DR. In the literature, researchers have focused on this automation using traditional machine learning-based algorithms and convolutional architectures. However, the past works hardly focused on essential parts of the retinal image to improve the model performance. In this study, we adopt and fine-tune transformer-based learning models to capture the crucial features of retinal images for a more nuanced understanding of DR severity. Additionally, we explore the effectiveness of image transformers to infer the degree of DR severity from fundus photographs. For experiments, we utilized the publicly available APTOS-2019 blindness detection dataset, where the performances of the transformer-based models were quite encouraging.

Motivation & Objective

• Motivate automated DR severity grading from fundus photographs to reduce inconsistencies in manual grading.
• Explore transformer-based architectures to capture salient retinal features for DR staging.
• Develop an ensemble of multiple image transformers to improve predictive performance on DR severity tasks.

Proposed method

• Preprocess fundus images with resizing, augmentations, and CLAHE to standardize inputs.
• Adapt and train four image transformers (ViT, BEiT, CaiT, DeiT) for DR severity classification.
• Ensemble the four transformers using weighted mean and majority voting to produce final predictions.
• Evaluate performance on the APTOS-2019 dataset with metrics including accuracy, kappa, precision, recall, F1, specificity, and balanced accuracy.
• Perform ablation and hyper-parameter analyses to assess contributions of individual transformers and MSA heads.

Figure 2: Workflow of ViT.

Experimental results

Research questions

• RQ1 Can transformer-based models effectively learn DR severity features from fundus images?
• RQ2 Does ensembling multiple image transformers yield superior performance over single models for DR severity grading?
• RQ3 What is the impact of preprocessing and hyper-parameters on DR severity classification with transformers?

Key findings

• Ensembled image transformers (EiT) achieve 94.63% accuracy with weighted mean and 91.26% with majority voting on APTOS-2019 test set.
• EiT with weighted mean attains a Cohen's kappa of 0.92 and a balanced accuracy of 95.75% (highest reported among configurations).
• Single CaiT transformer performed best among single-models before ensembling; ensemble methods improve beyond individual models.
• Across severity classes, EiT achieves high precision and recall for negative DR (class 0) and varying performance for other stages, with overall strong specificity.
• Hyper-parameter analysis shows increasing MSA heads up to 6 improves performance, and tuned alpha weights in weighted-mean ensemble yield peak accuracy of 94.63%.
• Compared to conventional CNN-based architectures and other transformers, EiT outperforms on accuracy, balanced accuracy, sensitivity, and specificity.

Figure 3: Internal view of a transformer encoder (TE).