[Paper Review] An Active Learning Method for Diabetic Retinopathy Classification with Uncertainty Quantification

Muhammad Ahtazaz Ahsan, Adnan Qayyum+2 more|arXiv (Cornell University)|Dec 24, 2020

Retinal Imaging and Analysis4 citations

TL;DR

This paper proposes a hybrid active learning framework for diabetic retinopathy classification that combines Bayesian convolutional neural networks (BCNNs) with uncertainty quantification and active learning to reduce reliance on labeled data. By using Monte Carlo dropout for uncertainty estimation and querying only the most informative, low-uncertainty samples, the method achieves state-of-the-art accuracy of 92% on binary classification and 92% on multi-class classification with minimal labeled data.

ABSTRACT

In recent years, deep learning (DL) techniques have provided state-of-the-art performance on different medical imaging tasks. However, the availability of good quality annotated medical data is very challenging due to involved time constraints and the availability of expert annotators, e.g., radiologists. In addition, DL is data-hungry and their training requires extensive computational resources. Another problem with DL is their black-box nature and lack of transparency on its inner working which inhibits causal understanding and reasoning. In this paper, we jointly address these challenges by proposing a hybrid model, which uses a Bayesian convolutional neural network (BCNN) for uncertainty quantification, and an active learning approach for annotating the unlabelled data. The BCNN is used as a feature descriptor and these features are then used for training a model, in an active learning setting. We evaluate the proposed framework for diabetic retinopathy classification problem and have achieved state-of-the-art performance in terms of different metrics.

Motivation & Objective

• To address the scarcity of high-quality, expert-annotated medical imaging data in diabetic retinopathy classification.
• To improve model reliability by quantifying prediction uncertainty using Bayesian deep learning.
• To reduce annotation cost and computational burden by leveraging active learning with uncertainty-based sampling.
• To develop a transparent, interpretable model that supports clinical decision-making through uncertainty-aware predictions.
• To evaluate the framework on both binary and multi-class diabetic retinopathy classification tasks.

Proposed method

• The framework employs a Bayesian convolutional neural network (BCNN) with Monte Carlo dropout to estimate predictive uncertainty and serve as a feature descriptor.
• Uncertainty is quantified via dropout-based Bayesian inference, where multiple forward passes estimate the posterior distribution over predictions.
• Active learning is implemented using two query strategies: pool-based sampling and query-by-committee, selecting samples with highest uncertainty for labeling.
• Only samples with low uncertainty (below a threshold) and consistent predictions across MC dropout runs are selected for labeling to reduce error propagation.
• The BCNN features are used to train a downstream classifier in an active learning loop, iteratively improving performance with minimal labeled data.
• The model is trained from scratch without ImageNet pretraining to ensure task-specific generalization in medical imaging.

Experimental results

Research questions

• RQ1 Can Bayesian deep learning improve uncertainty quantification in diabetic retinopathy classification compared to deterministic models?
• RQ2 How effective is active learning with uncertainty sampling in reducing the number of required labeled samples for DR classification?
• RQ3 Can a hybrid BCNN-active learning framework achieve state-of-the-art performance while maintaining model interpretability?
• RQ4 How does uncertainty quantification affect model confidence and reliability in clinical decision support?
• RQ5 What is the impact of hyperparameter tuning on uncertainty estimation and model performance in BCNNs for medical imaging?

Key findings

• The BCNN achieved a binary classification accuracy of 92% for low-confidence predictions and 81% for high-confidence predictions, demonstrating effective uncertainty-aware inference.
• For multi-class classification, the BCNN achieved 92% accuracy under high-confidence predictions, indicating strong performance with uncertainty calibration.
• The active learning framework improved state-of-the-art results, achieving 91% accuracy in binary classification with reduced labeled data.
• The integration of uncertainty quantification reduced model overconfidence and improved reliability in ambiguous or complex cases.
• The approach outperformed deterministic CNNs in both accuracy and uncertainty estimation, especially in low-data regimes.
• Hyperparameter tuning of dropout rate and uncertainty threshold significantly influenced model stability and performance, suggesting room for further optimization.