[Paper Review] Pi-PE: A Pipeline for Pulmonary Embolism Detection using Sparsely Annotated 3D CT Images

Deepta Rajan, David Beymer+2 more|arXiv (Cornell University)|Oct 4, 2019

Venous Thromboembolism Diagnosis and Management10 citations

TL;DR

This paper proposes a two-stage, 2D CNN-based pipeline for pulmonary embolism (PE) detection in 3D CT scans using only sparse annotations (every 10 mm), avoiding costly dense labeling. It combines a context-augmented U-Net for mask generation and a 2D Conv-LSTM with multiple instance learning for classification, achieving state-of-the-art AUC of 0.85 on high-severity PEs with only 3.17 million parameters.

ABSTRACT

Pulmonary embolisms (PE) are known to be one of the leading causes for cardiac-related mortality. Due to inherent variabilities in how PE manifests and the cumbersome nature of manual diagnosis, there is growing interest in leveraging AI tools for detecting PE. In this paper, we build a two-stage detection pipeline that is accurate, computationally efficient, robust to variations in PE types and kernels used for CT reconstruction, and most importantly, does not require dense annotations. Given the challenges in acquiring expert annotations in large-scale datasets, our approach produces state-of-the-art results with very sparse emboli contours (at 10mm slice spacing), while using models with significantly lower number of parameters. We achieve AUC scores of 0.94 on the validation set and 0.85 on the test set of highly severe PEs. Using a large, real-world dataset characterized by complex PE types and patients from multiple hospitals, we present an elaborate empirical study and provide guidelines for designing highly generalizable pipelines.

Motivation & Objective

• To develop an accurate, efficient, and robust PE detection pipeline that does not require dense annotations, which are costly and time-consuming to obtain in clinical settings.
• To address the challenge of detecting subtle, subsegmental PEs that are difficult to identify due to their asymmetric and irregular appearance in pulmonary vasculature.
• To design a generalizable pipeline that performs well across diverse imaging protocols and reconstruction kernels used in multi-hospital real-world datasets.
• To reduce model complexity and parameter count while maintaining high performance, in contrast to large 3D CNNs requiring extensive pre-training.
• To provide empirical guidelines for building effective, scalable, and robust medical image analysis pipelines for rare or hard-to-detect pathologies like PE.

Proposed method

• A two-stage pipeline is used: Stage 1 employs a context-augmented U-Net to generate segmentation masks from sparse annotations, improving localization accuracy.
• The input CT volume and predicted mask are multiplied to produce a masked volume, which is then fed into Stage 2 for classification.
• Stage 2 uses a 2D Conv-LSTM model to process the sequence of 2D slices, modeling each 3D volume as a bag of instances.
• Multiple instance learning (MIL) is applied by aggregating features across slices, enabling the model to detect PE based on the presence of any positive instance.
• The model is trained using a focal loss to handle class imbalance, improving performance on rare positive cases.
• Context augmentation in the U-Net enhances feature learning by incorporating spatial and contextual information from neighboring slices.

Experimental results

Research questions

• RQ1 Can a two-stage 2D CNN pipeline achieve state-of-the-art PE detection performance without requiring dense annotations?
• RQ2 How does the proposed method generalize across diverse CT reconstruction kernels and multi-hospital datasets with varying imaging protocols?
• RQ3 What is the impact of model architecture, parameter count, and training loss on detection performance and robustness?
• RQ4 How does the performance vary across different PE types, especially subsegmental and segmental PEs that are harder to detect?
• RQ5 To what extent does context augmentation and MIL improve detection in low-data regimes?

Key findings

• The pipeline achieves an AUC of 0.94 on the validation set and 0.85 on the test set for high-severity PEs, demonstrating state-of-the-art performance.
• The model achieves an AUC of 0.70 for low-severity PEs such as subsegmental and segmental types, indicating detectability despite their subtlety.
• The model is robust to variations in CT reconstruction kernels, maintaining an AUC of 0.78 across all kernel types, including non-standard ones.
• The model uses only 3.17 million parameters—significantly fewer than the 28.4 million in the prior state-of-the-art PENet model—while outperforming it on diverse, real-world data.
• Increasing the number of instances (T) from 25 to 100 in the MIL setting leads to a steady improvement in performance, indicating the benefit of longer sequence modeling.
• The use of focal loss instead of binary cross-entropy loss improves performance on imbalanced datasets, particularly for rare PE cases.