[Paper Review] CheXNet: Radiologist-Level Pneumonia Detection on Chest X-Rays with Deep Learning

Pranav Rajpurkar, Jeremy Irvin+10 more|arXiv (Cornell University)|Nov 14, 2017

COVID-19 diagnosis using AIMedicine1,447 citations

TL;DR

CheXNet, a 121-layer DenseNet trained on ChestX-ray14, detects pneumonia in chest X-rays at radiologist-competitive to superior performance and extends to all 14 diseases with state-of-the-art results. It also provides heatmaps (CAMs) to localize pathology.

ABSTRACT

We develop an algorithm that can detect pneumonia from chest X-rays at a level exceeding practicing radiologists. Our algorithm, CheXNet, is a 121-layer convolutional neural network trained on ChestX-ray14, currently the largest publicly available chest X-ray dataset, containing over 100,000 frontal-view X-ray images with 14 diseases. Four practicing academic radiologists annotate a test set, on which we compare the performance of CheXNet to that of radiologists. We find that CheXNet exceeds average radiologist performance on the F1 metric. We extend CheXNet to detect all 14 diseases in ChestX-ray14 and achieve state of the art results on all 14 diseases.

Motivation & Objective

• Develop an automated pneumonia detection model for chest X-rays that exceeds practicing radiologists.
• Leverage the ChestX-ray14 dataset to train a deep network for multiple thoracic pathologies.
• Provide localization heatmaps to indicate regions contributing to predictions.

Proposed method

• Use a 121-layer DenseNet pretrained on ImageNet with a single sigmoid output for pneumonia presence.
• Optimize with weighted binary cross-entropy to handle class imbalance (weights w+ and w−).
• Train end-to-end using Adam with learning-rate scheduling based on validation loss.
• Extend the model to output a 14-class vector with independent binary cross-entropy losses for all pathologies.
• Evaluate with F1 score against radiologists on a 420-image test set using bootstrap CIs.

Experimental results

Research questions

• RQ1 Can a deep CNN achieve radiologist-level performance for pneumonia detection on chest X-rays?
• RQ2 Does extending the model to 14 pathologies yield state-of-the-art AUROC across all classes?
• RQ3 Can class activation maps meaningfully localize pneumonia-related regions in X-ray images?

Key findings

• CheXNet achieves F1 score of 0.435 (95% CI 0.387, 0.481), higher than the radiologist average of 0.387 (95% CI 0.330, 0.442).
• CheXNet’s pneumonia detection performance is statistically significantly higher than radiologists (difference 0.051, 95% CI 0.005, 0.084).
• On ChestX-ray14, CheXNet outperforms previous state-of-the-art across all 14 pathology classes (per-class AUROC).
• CheXNet shows particularly large improvements for Mass, Nodule, Pneumonia, and Emphysema (AUROC gains >0.05).
• The model provides class activation maps to localize regions contributing to predictions.
• CheXNet was trained on 112,120 images, with 420-test radiologist-evaluated set for pneumonia.