[Paper Review] DeepLung: Deep 3D Dual Path Nets for Automated Pulmonary Nodule Detection and Classification

Wentao Zhu, Chaochun Liu+2 more|arXiv (Cornell University)|Jan 25, 2018

Lung Cancer Diagnosis and Treatment30 references64 citations

TL;DR

DeepLung is a fully automated lung CT cancer diagnosis system that detects nodules with a 3D Faster R-CNN using 3D dual path nets and classifies them with GBM based on deep features, nodule size, and raw pixels.

ABSTRACT

In this work, we present a fully automated lung computed tomography (CT) cancer diagnosis system, DeepLung. DeepLung consists of two components, nodule detection (identifying the locations of candidate nodules) and classification (classifying candidate nodules into benign or malignant). Considering the 3D nature of lung CT data and the compactness of dual path networks (DPN), two deep 3D DPN are designed for nodule detection and classification respectively. Specifically, a 3D Faster Regions with Convolutional Neural Net (R-CNN) is designed for nodule detection with 3D dual path blocks and a U-net-like encoder-decoder structure to effectively learn nodule features. For nodule classification, gradient boosting machine (GBM) with 3D dual path network features is proposed. The nodule classification subnetwork was validated on a public dataset from LIDC-IDRI, on which it achieved better performance than state-of-the-art approaches and surpassed the performance of experienced doctors based on image modality. Within the DeepLung system, candidate nodules are detected first by the nodule detection subnetwork, and nodule diagnosis is conducted by the classification subnetwork. Extensive experimental results demonstrate that DeepLung has performance comparable to experienced doctors both for the nodule-level and patient-level diagnosis on the LIDC-IDRI dataset.\footnote{https://github.com/uci-cbcl/DeepLung.git}

Motivation & Objective

• Motivate automated, end-to-end lung CT cancer diagnosis to reduce reliance on manual analysis by clinicians.
• Develop a compact, effective 3D deep learning architecture tailored to the 3D nature of CT data for detection and classification.
• Integrate nodule detection and classification into a coherent system with patient-level diagnosis via nodule fusion.
• Leverage 3D dual path networks to improve performance while reducing parameter count compared to 3D ResNets.

Proposed method

• Design a 3D Faster R-CNN with 3D dual path blocks and a U-net-like encoder-decoder for nodule detection on CT volumes.
• Use a deep 3D dual path network to extract features from cropped nodules for classification.
• Train a gradient boosting machine (GBM) that combines deep features, nodule size, and raw nodule pixels for benign/malignant classification.
• Crop detected nodules to 32x32x32 for feature extraction, and use 16x16x16 pixel crops as additional inputs to GBM.
• Evaluate detection with FROC on LUNA16 and classification with accuracy on LIDC-IDRI, comparing to state-of-the-art and experienced doctors.

Experimental results

Research questions

• RQ1 Can a fully automated 3D DeepLung pipeline achieve nodule-level accuracy comparable to or surpassing experienced radiologists on public CT datasets?
• RQ2 Does a 3D dual path network provide advantages in detection and classification over traditional 3D CNNs for pulmonary nodules?
• RQ3 Does combining deep features with nodule size and raw pixel data via GBM yield superior nodule classification performance?
• RQ4 Can fusion of nodule-level classifications produce competitive patient-level ( CT) cancer diagnoses?

Key findings

• 3D DPN-based detector achieves higher FROC than a 3D ResNet-based detector with far fewer parameters (84.2% vs 83.4%, using 1/4 of the parameters).
• Nodule classification with deep 3D DPN features plus GBM and size/pixel inputs achieves 90.44% accuracy on LIDC-IDRI when combining all features.
• DeepLung’s nodule-level predictions on confident nodules exceed average expert doctor performance (average 92.74% vs doctors’ 91.25%).
• On fully automated patient-level diagnosis, DeepLung attains 81.41% accuracy, close to the average doctor performance (82.31%), and better than one doctor in isolation.
• GBM reduces false positives substantially (97.02% FP reduction on FP detected nodules), while maintaining solid TP performance.
• Kappa agreement for DeepLung on patient-level diagnosis is 63.02%, comparable to doctors’ average 64.46%.