[Paper Review] Finding Covid-19 from Chest X-rays using Deep Learning on a Small Dataset

Lawrence Hall, Rahul Paul+2 more|arXiv (Cornell University)|Apr 5, 2020

COVID-19 diagnosis using AI7 citations

TL;DR

This study proposes a deep learning approach to detect COVID-19 from chest X-rays using a small dataset of 102 COVID-19 and 102 non-COVID pneumonia cases. Fine-tuning a pretrained convolutional neural network achieved 100% sensitivity on unseen COVID-19 cases and an AUC of 0.997, demonstrating high potential despite limited data and methodological limitations.

ABSTRACT

Testing for COVID-19 has been unable to keep up with the demand. Further, the false negative rate is projected to be as high as 30% and test results can take some time to obtain. X-ray machines are widely available and provide images for diagnosis quickly. This paper explores how useful chest X-ray images can be in diagnosing COVID-19 disease. We have obtained 122 chest X-rays of COVID-19 and over 4,000 chest X-rays of viral and bacterial pneumonia. A pretrained deep convolutional neural network has been tuned on 102 COVID-19 cases and 102 other pneumonia cases in a 10-fold cross validation. The results were all 102 COVID-19 cases were correctly classified and there were 8 false positives resulting in an AUC of 0.997. On a test set of 20 unseen COVID-19 cases all were correctly classified and more than 95% of 4171 other pneumonia examples were correctly classified. This study has flaws, most critically a lack of information about where in the disease process the COVID-19 cases were and the small data set size. More COVID-19 case images will enable a better answer to the question of how useful chest X-rays can be for diagnosing COVID-19 (so please send them).

Motivation & Objective

• To investigate the feasibility of using chest X-rays for early detection of COVID-19 when diagnostic testing is insufficient or delayed.
• To evaluate the performance of deep learning models on a small, imbalanced dataset of COVID-19 and non-COVID pneumonia X-rays.
• To determine whether transfer learning with a pretrained CNN can achieve high diagnostic accuracy despite limited training data.
• To assess generalization performance on unseen COVID-19 and non-COVID pneumonia cases.

Proposed method

• A pretrained deep convolutional neural network was fine-tuned on a curated dataset of 102 COVID-19 and 102 non-COVID pneumonia chest X-rays.
• 10-fold cross-validation was used to evaluate model performance on the training data.
• The model was tested on a separate set of 20 unseen COVID-19 X-rays and 4,171 non-COVID pneumonia X-rays.
• Transfer learning was employed to adapt the pretrained network to the specific task of distinguishing COVID-19 from other pulmonary infections.
• Model evaluation focused on sensitivity, specificity, and area under the ROC curve (AUC).

Experimental results

Research questions

• RQ1 Can deep learning models achieve high diagnostic accuracy for COVID-19 detection using only a small number of chest X-ray images?
• RQ2 How does transfer learning perform on a small, imbalanced dataset of COVID-19 and non-COVID pneumonia X-rays?
• RQ3 What is the generalization performance of the model on unseen, real-world cases of COVID-19 and other pneumonia types?
• RQ4 How do false positive rates and sensitivity compare across different test sets?

Key findings

• All 102 COVID-19 cases in the training set were correctly classified, achieving 100% sensitivity.
• The model achieved an area under the ROC curve (AUC) of 0.997 on the 10-fold cross-validation.
• On a test set of 20 unseen COVID-19 cases, all were correctly classified, confirming high sensitivity in external validation.
• More than 95% of the 4,171 non-COVID pneumonia cases were correctly classified, indicating strong specificity.
• The model produced 8 false positives across the entire test set, highlighting a low but non-zero false positive rate.