[Paper Review] Large Scale Automated Reading of Frontal and Lateral Chest X-Rays using Dual Convolutional Neural Networks

Jonathan Rubin, Deepan Sanghavi+4 more|arXiv (Cornell University)|Apr 20, 2018

COVID-19 diagnosis using AI10 references77 citations

TL;DR

The paper trains CNNs on the large MIMIC-CXR dataset to recognize 14 thoracic findings from frontal and lateral chest X-rays and introduces DualNet to jointly process paired frontal and lateral views, improving performance over independent models.

ABSTRACT

The MIMIC-CXR dataset is (to date) the largest released chest x-ray dataset consisting of 473,064 chest x-rays and 206,574 radiology reports collected from 63,478 patients. We present the results of training and evaluating a collection of deep convolutional neural networks on this dataset to recognize multiple common thorax diseases. To the best of our knowledge, this is the first work that trains CNNs for this task on such a large collection of chest x-ray images, which is over four times the size of the largest previously released chest x-ray corpus (ChestX-Ray14). We describe and evaluate individual CNN models trained on frontal and lateral CXR view types. In addition, we present a novel DualNet architecture that emulates routine clinical practice by simultaneously processing both frontal and lateral CXR images obtained from a radiological exam. Our DualNet architecture shows improved performance in recognizing findings in CXR images when compared to applying separate baseline frontal and lateral classifiers.

Motivation & Objective

• Demonstrate large-scale automatic thorax disease classification on the MIMIC-CXR dataset.
• Evaluate frontal (PA/AP) and lateral view-specific CNNs for multi-label classification.
• Propose and assess the DualNet architecture that processes both frontal and lateral X-rays concurrently.
• Compare DualNet against separate frontal and lateral classifiers to quantify performance gains.

Proposed method

• Train separate DenseNet-121-based CNNs (1-channel inputs) for PA, AP, and Lateral views using multi-label binary cross-entropy.
• Use 12–128 networks with a global average pooling and a final 14-class sigmoid output.
• Apply NegBio-derived labels from radiology reports to map to 14 thoracic findings (or No Finding).
• Process images with 512x512 resizing while preserving aspect ratio and normalize to [0,1].
• Train DualNet variants that fuse frontal and lateral networks by concatenating their outputs before the final classifier.
• Evaluate using AUC per class on a subject-split test set; compare DualNet to individual-view baselines.

Experimental results

Research questions

• RQ1 Can CNNs trained on the MIMIC-CXR dataset accurately identify common thoracic diseases from frontal and lateral chest X-rays?
• RQ2 Does integrating frontal and lateral views via a DualNet architecture improve classification performance over independent view-specific models?
• RQ3 How do PA, AP, and Lateral views compare in detection performance for specific findings?
• RQ4 What is the impact of using paired frontal-lateral data on multi-label thorax disease detection?

Key findings

• PA models achieve highest average AUC among frontal views (0.702) vs AP (0.655).
• Lateral view models achieve the highest average AUC among single-view models (0.706).
• DualNet consistently outperforms separate frontal and lateral classifiers in most PA+Lateral (12/14) and AP+Lateral (10/14) cases.
• For PA+Lateral, DualNet average AUC = 0.721 vs 0.690 for independent models.
• For AP+Lateral, DualNet average AUC = 0.668 vs 0.637 for independent models.
• Overall, dual-view joint processing yields robust gains across several findings.