[Paper Review] CovidAID: COVID-19 Detection Using Chest X-Ray

Arpan Mangal, Surya Kalia+5 more|arXiv (Cornell University)|Apr 21, 2020

COVID-19 diagnosis using AI9 references132 citations

TL;DR

CovidAID uses a pre-trained CheXNet backbone to classify chest X-rays into Normal, Bacterial Pneumonia, Viral Pneumonia, and COVID-19 (also a 3-class variant). It achieves near-perfect COVID-19 recall and outperforms COVID-Net on the same dataset."

ABSTRACT

The exponential increase in COVID-19 patients is overwhelming healthcare systems across the world. With limited testing kits, it is impossible for every patient with respiratory illness to be tested using conventional techniques (RT-PCR). The tests also have long turn-around time, and limited sensitivity. Detecting possible COVID-19 infections on Chest X-Ray may help quarantine high risk patients while test results are awaited. X-Ray machines are already available in most healthcare systems, and with most modern X-Ray systems already digitized, there is no transportation time involved for the samples either. In this work we propose the use of chest X-Ray to prioritize the selection of patients for further RT-PCR testing. This may be useful in an inpatient setting where the present systems are struggling to decide whether to keep the patient in the ward along with other patients or isolate them in COVID-19 areas. It would also help in identifying patients with high likelihood of COVID with a false negative RT-PCR who would need repeat testing. Further, we propose the use of modern AI techniques to detect the COVID-19 patients using X-Ray images in an automated manner, particularly in settings where radiologists are not available, and help make the proposed testing technology scalable. We present CovidAID: COVID-19 AI Detector, a novel deep neural network based model to triage patients for appropriate testing. On the publicly available covid-chestxray-dataset [2], our model gives 90.5% accuracy with 100% sensitivity (recall) for the COVID-19 infection. We significantly improve upon the results of Covid-Net [10] on the same dataset.

Motivation & Objective

• Motivate triage of COVID-19 testing when RT-PCR resources are limited.
• Leverage Chest X-Ray data to prioritize RT-PCR testing in SARI patients.
• Develop a deep learning model based on CheXNet to classify chest X-Rays into four (or three) clinically relevant classes.
• Provide open access to training code and models to accelerate research in AI-assisted COVID-19 detection.

Proposed method

• Use a pre-trained CheXNet DenseNet-121 backbone.
• Replace the final classifier to output four (or three) classes with sigmoid activations.
• Train in two stages: (i) train only the final layer with backbone frozen, (ii) fine-tune end-to-end.
• Employ a weighted binary cross-entropy loss per class with class-specific weights w_c^+ and w_c^- to handle class imbalance.
• In each batch, sample pneumonia data sparsely to prevent COVID-19 signal being masked (ratio 5:5:5:1 for 4-class; 7:7:1 for 3-class).
• Evaluate on a split of the covid-chestxray-dataset and chest-xray-pneumonia datasets; use per-patient splits to avoid leakage.

Experimental results

Research questions

• RQ1 Can a DenseNet-based model trained with CheXNet weights accurately distinguish COVID-19 from normal and other pneumonias in chest X-rays?
• RQ2 Does the four-class configuration (Normal, Bacterial Pneumonia, Viral Pneumonia, COVID-19) improve COVID-19 triage compared to a three-class setup?
• RQ3 How does CovidAID compare to the COVID-Net baseline on the same test split?
• RQ4 Is the model's attention/correlation with radiologist-visible features supported by saliency analyses?

Key findings

• COVID-19 class achieves AUROC up to 0.9997 with sensitivity 1.000 and PPV 0.968 in the three-class configuration.
• Four-class configuration yields AUROC for COVID-19 of 0.9994 with sensitivity 1.000 and PPV 0.938; overall 4-class accuracy 87.2%.
• Three-class configuration yields overall accuracy 90.5% with class-wise AUROCs: Normal Lung 0.9795, Pneumonia 0.9814, COVID-19 0.9997; sensitivities and PPVs: 0.744/0.989 (Normal), 0.995/0.868 (Pneumonia), 1.000/0.938 (COVID-19).
• Compared to COVID-Net, CovidAID outperforms by more than 0.14 AUROC in detecting Pneumonia and COVID-19 on the same test split.
• Saliency maps using RISE corroborate that model focuses on semantically meaningful lung regions for COVID-19 predictions.