[Paper Review] Abnormality Detection and Localization in Chest X-Rays using Deep Convolutional Neural Networks

Mohammad Tariqul Islam, Abdul Aowal+2 more|arXiv (Cornell University)|May 27, 2017

COVID-19 diagnosis using AI25 references115 citations

TL;DR

The paper evaluates multiple deep convolutional networks on chest X-ray abnormalities using Indiana, JSRT, and Shenzhen datasets, shows ensemble DCN models outperform rule-based methods, and introduces occlusion-based localization for cardiomegaly and pulmonary edema.

ABSTRACT

Chest X-Rays (CXRs) are widely used for diagnosing abnormalities in the heart and lung area. Automatically detecting these abnormalities with high accuracy could greatly enhance real world diagnosis processes. Lack of standard publicly available dataset and benchmark studies, however, makes it difficult to compare various detection methods. In order to overcome these difficulties, we have used a publicly available Indiana CXR, JSRT and Shenzhen dataset and studied the performance of known deep convolutional network (DCN) architectures on different abnormalities. We find that the same DCN architecture doesn't perform well across all abnormalities. Shallow features or earlier layers consistently provide higher detection accuracy compared to deep features. We have also found ensemble models to improve classification significantly compared to single model. Combining these insight, we report the highest accuracy on chest X-Ray abnormality detection on these datasets. We find that for cardiomegaly detection, the deep learning method improves the accuracy by a staggering 17 percentage point compared to rule based methods. We applied the techniques to the problem of tuberculosis detection on a different dataset and achieved the highest accuracy. Our localization experiments using these trained classifiers show that for spatially spread out abnormalities like cardiomegaly and pulmonary edema, the network can localize the abnormalities successfully most of the time. One remarkable result of the cardiomegaly localization is that the heart and its surrounding region is most responsible for cardiomegaly detection, in contrast to the rule based models where the ratio of heart and lung area is used as the measure. We believe that through deep learning based classification and localization, we will discover many more interesting features in medical image diagnosis that are not considered traditionally.

Motivation & Objective

• Evaluate the performance of various DCN architectures (AlexNet, VGG, ResNet) for chest X-ray abnormality detection across public datasets.
• Assess the benefit of using ensemble models over single DCNs for cardiomegaly and other abnormalities.
• Develop and apply a localization method based on occlusion sensitivity to identify regions contributing to classifier decisions.
• Compare DCN-based cardiomegaly detection with rule-based features and report improvements.
• Demonstrate generalization to another dataset for tuberculosis detection.

Proposed method

• Fine-tune pre-trained DCNs (AlexNet, VGG-16/19, ResNet-50/101/152) on chest X-ray datasets using Adam optimizer (lr=0.001).
• Extract features from selected layers (e.g., res4f, res4b22, res4b35; pool5), and train binary classifiers for abnormality detection.
• Explore ensemble predictions via simple linear averaging of model probabilities across multiple DCNs.
• Use occlusion sensitivity with 40x40 patches to generate localization probability maps by sliding the occluder over CXRs and tracking drops in pathology probability.
• Evaluate using accuracy, AUC, sensitivity, specificity on balanced train/test splits and multiple random splits for robustness.
• Apply the approach to Shenzhen TB dataset to demonstrate transfer.

Experimental results

Research questions

• RQ1 How do different DCN architectures perform on cardiomegaly and other chest X-ray abnormalities across public datasets?
• RQ2 Does an ensemble of DCNs improve detection accuracy and AUC compared to single models?
• RQ3 Can occlusion-based localization reliably localize abnormalities such as cardiomegaly and pulmonary edema in CXRs?
• RQ4 How does DCN-based cardiomegaly detection compare to rule-based methods, and does the approach generalize to TB detection on Shenzhen dataset?

Key findings

• Ensemble DCN models achieve higher cardiomegaly detection accuracy (up to 93.0%) and AUC (up to 0.9728) than single models.
• Single-model performance varies by abnormality; for cardiomegaly, VGG-19 yields high accuracy (92%) and AUC (0.94), while ResNet-152 offers strong mean performance across random splits.
• Shallow features from earlier layers often outperform deep features for cardiomegaly detection; features from ResNet-152 early layers (e.g., res4b35) provide better accuracy.
• Dropping dropout generally helps deeper models but can reduce performance for deeper networks; an ensemble of six DCNs provides the best overall performance (accuracy 93%, AUC 0.97) for cardiomegaly.
• Occlusion-based localization highlights the heart region as primary for cardiomegaly detection and lung regions for pulmonary edema; localization results align with clinically expected regions, though some results are outside expected anatomical confines for edema.
• On the Shenzhen TB dataset, the ensemble model achieves highest accuracy (90.0%) and AUC (0.94).