[Paper Review] Deep-learning models in medical image analysis: Detection of esophagitis from the Kvasir Dataset

Kyoka Yoshiok, Kensuke Tanioka+2 more|arXiv (Cornell University)|Jan 6, 2023

Esophageal Cancer Research and Treatment4 citations

TL;DR

This study evaluates four deep learning models—GoogLeNet, ResNet-50, MobileNet V2, and MobileNet V3—for detecting esophagitis in endoscopic images from the Kvasir dataset. GoogLeNet achieved the highest F1-score (0.843), while MobileNet V3 showed the highest true positive rate (0.950) and precision (0.901), indicating superior confidence and accuracy in esophagitis detection. Explainability techniques like Grad-CAM and SHAP revealed model attention to clinically relevant mucosal tears.

ABSTRACT

Early detection of esophagitis is important because this condition can progress to cancer if left untreated. However, the accuracies of different deep learning models in detecting esophagitis have yet to be compared. Thus, this study aimed to compare the accuracies of convolutional neural network models (GoogLeNet, ResNet-50, MobileNet V2, and MobileNet V3) in detecting esophagitis from the open Kvasir dataset of endoscopic images. Results showed that among the models, GoogLeNet achieved the highest F1-scores. Based on the average of true positive rate, MobileNet V3 predicted esophagitis more confidently than the other models. The results obtained using the models were also compared with those obtained using SHapley Additive exPlanations and Gradient-weighted Class Activation Mapping.

Motivation & Objective

• To compare the diagnostic accuracy of four state-of-the-art CNN models—GoogLeNet, ResNet-50, MobileNet V2, and MobileNet V3—in detecting esophagitis from endoscopic images.
• To evaluate model performance using standard metrics (accuracy, precision, recall, F1-score) on the open Kvasir dataset.
• To analyze model interpretability using Grad-CAM and SHAP to assess whether models focus on clinically relevant features such as mucosal tears.
• To determine which model offers the best balance of accuracy, computational efficiency, and clinical confidence for esophagitis detection.

Proposed method

• Trained four pre-trained CNN models—GoogLeNet, ResNet-50, MobileNet V2, and MobileNet V3—on the Kvasir endoscopic image dataset for binary classification of esophagitis vs. z-line.
• Used standard evaluation metrics: accuracy, precision, recall, specificity, and F1-score derived from confusion matrix values (TP, FP, TN, FN).
• Applied Grad-CAM to visualize attention maps highlighting image regions most influential for classification decisions.
• Utilized SHAP to compute feature-level contributions, identifying which image patches support or reject the prediction of esophagitis.
• Compared model predictions with ground truth and analyzed false negatives to assess model reliability and interpretability.
• Evaluated model confidence via average true positive rate across test images to assess prediction consistency.

Experimental results

Research questions

• RQ1 Which deep learning model achieves the highest F1-score in detecting esophagitis from endoscopic images in the Kvasir dataset?
• RQ2 How do the precision and true positive rate of MobileNet V3 compare to other models in classifying esophagitis?
• RQ3 To what extent do Grad-CAM and SHAP explain the decision-making process of the models, and which method better localizes clinically relevant features?
• RQ4 Do the models attend to anatomical features such as mucosal tears relative to the z-line, as required for accurate diagnosis?

Key findings

• GoogLeNet achieved the highest F1-score of 0.843, indicating the most balanced precision and recall among all models.
• MobileNet V3 recorded the highest average true positive rate of 0.950, demonstrating the most confident detection of esophagitis.
• MobileNet V3 also achieved the highest precision (0.901) and specificity (0.908), indicating strong reliability in positive predictions.
• SHAP analysis revealed that MobileNet V3 identified multiple mucosal tears more accurately than Grad-CAM, especially in high-confidence predictions.
• In false negative cases, SHAP correctly highlighted inflammatory regions as negative contributors to the z-line prediction, suggesting model awareness of disease features despite misclassification.
• Grad-CAM showed gradient saturation in false negative cases, leading to diffuse activation and reduced interpretability compared to SHAP.