[Paper Review] TRk-CNN: Transferable Ranking-CNN for image classification of glaucoma, glaucoma suspect, and normal eyes

Tae Joon Jun, Youngsub Eom+5 more|arXiv (Cornell University)|May 16, 2019

Retinal Imaging and Analysis37 references4 citations

TL;DR

This paper proposes TRk-CNN, a transferable ranking convolutional neural network that improves classification of glaucoma, glaucoma suspect, and normal eyes by modeling the ordinal relationship between disease stages. By combining primitive classification scores with learned weights, TRk-CNN achieves 92.96% average accuracy, outperforming both standard multi-class CNN and Ranking-CNN by 8.04% and 9.54% respectively, with a 26.83% improvement in sensitivity for glaucoma suspects.

ABSTRACT

In this paper, we proposed Transferable Ranking Convolutional Neural Network (TRk-CNN) that can be effectively applied when the classes of images to be classified show a high correlation with each other. The multi-class classification method based on the softmax function, which is generally used, is not effective in this case because the inter-class relationship is ignored. Although there is a Ranking-CNN that takes into account the ordinal classes, it cannot reflect the inter-class relationship to the final prediction. TRk-CNN, on the other hand, combines the weights of the primitive classification model to reflect the inter-class information to the final classification phase. We evaluated TRk-CNN in glaucoma image dataset that was labeled into three classes: normal, glaucoma suspect, and glaucoma eyes. Based on the literature we surveyed, this study is the first to classify three status of glaucoma fundus image dataset into three different classes. We compared the evaluation results of TRk-CNN with Ranking-CNN (Rk-CNN) and multi-class CNN (MC-CNN) using the DenseNet as the backbone CNN model. As a result, TRk-CNN achieved an average accuracy of 92.96%, specificity of 93.33%, sensitivity for glaucoma suspect of 95.12% and sensitivity for glaucoma of 93.98%. Based on average accuracy, TRk-CNN is 8.04% and 9.54% higher than Rk-CNN and MC-CNN and surprisingly 26.83% higher for sensitivity for suspicious than multi-class CNN. Our TRk-CNN is expected to be effectively applied to the medical image classification problem where the disease state is continuous and increases in the positive class direction.

Motivation & Objective

• To address the limitation of standard multi-class CNNs in modeling ordinal relationships between medical image classes, such as normal, glaucoma suspect, and glaucoma.
• To improve classification performance in medical imaging tasks where disease states progress continuously along a positive direction.
• To develop a method that effectively incorporates inter-class relationships into the final prediction phase, overcoming the shortcomings of existing Ranking-CNN approaches.
• To demonstrate the effectiveness of TRk-CNN on a novel three-class glaucoma fundus image dataset collected from Korea University Medical Center.

Proposed method

• TRk-CNN employs a two-stage pipeline: primitive classification using N−1 binary classifiers to predict ordinal relationships between classes.
• Each primitive classifier generates a Class Activation Map (CAM) to localize discriminative regions, such as the optic disc and cup.
• The model aggregates predictions from all primitive classifiers by learning weighted fusion of their outputs, preserving inter-class ordinal information.
• A DenseNet backbone is used for feature extraction, with fully-connected layers removed from the binary sub-classifiers to maintain spatial localization.
• The final classification score is computed as a weighted sum of primitive classifier outputs, enabling end-to-end training with backpropagation.
• The method is trained and evaluated on a real-world fundus image dataset with three clinically relevant classes: normal, glaucoma suspect, and glaucoma.

Experimental results

Research questions

• RQ1 Can a deep learning model that explicitly models ordinal relationships between disease stages improve classification performance in medical image analysis?
• RQ2 How does TRk-CNN compare to standard multi-class CNN and Ranking-CNN in classifying glaucoma with three progressive stages?
• RQ3 To what extent does incorporating inter-class relationships through learned weight fusion enhance sensitivity for intermediate disease states like glaucoma suspect?
• RQ4 Does the proposed method generalize well across different regions of interest (e.g., disc, cup, and original fundus regions) in glaucoma images?
• RQ5 Can TRk-CNN achieve superior performance on a real-world glaucoma fundus dataset with clinically meaningful three-class labeling?

Key findings

• TRk-CNN achieved an average accuracy of 92.96%, significantly outperforming Rk-CNN (84.92%) and MC-CNN (83.42%) by 8.04 and 9.54 percentage points respectively.
• The sensitivity for glaucoma suspect classification reached 95.12%, a 26.83% improvement over the multi-class CNN baseline.
• Specificity was 93.33%, indicating strong performance in correctly identifying non-diseased cases.
• The model demonstrated robust feature activation in the optic disc and cup regions, even in deeper convolutional layers, confirming effective localization of disease-relevant anatomy.
• Visualizations showed that the model automatically emphasized pathological regions like the cup, even when only the disc region was manually annotated.
• The best-performing sub-models did not always yield the best final results, highlighting the importance of ensemble weighting in the final classification stage.