[Paper Review] Atrial Septal Defect Detection in Children Based on Ultrasound Video Using Multiple Instances Learning

Yiman Liu, Qiming Huang+10 more|arXiv (Cornell University)|Jun 6, 2023

Cardiovascular and Diving-Related ComplicationsMedicine3 citations

TL;DR

This paper proposes a multiple instance learning-based deep learning framework for detecting atrial septal defects (ASD) in pediatric ultrasound videos using two standard echocardiographic views—subcostal and low parasternal four-chamber. The method employs ResNet18 and 3D-CNN (R3D) for spatiotemporal feature extraction, combined with block random sampling and maximal agreement decision-making, achieving 84.95% accuracy and 89.33% AUC in five-fold cross-validation on a private dataset of 300 children.

ABSTRACT

Purpose: Congenital heart defect (CHD) is the most common birth defect. Thoracic echocardiography (TTE) can provide sufficient cardiac structure information, evaluate hemodynamics and cardiac function, and is an effective method for atrial septal defect (ASD) examination. This paper aims to study a deep learning method based on cardiac ultrasound video to assist in ASD diagnosis. Materials and methods: We select two standard views of the atrial septum (subAS) and low parasternal four-compartment view (LPS4C) as the two views to identify ASD. We enlist data from 300 children patients as part of a double-blind experiment for five-fold cross-validation to verify the performance of our model. In addition, data from 30 children patients (15 positives and 15 negatives) are collected for clinician testing and compared to our model test results (these 30 samples do not participate in model training). We propose an echocardiography video-based atrial septal defect diagnosis system. In our model, we present a block random selection, maximal agreement decision and frame sampling strategy for training and testing respectively, resNet18 and r3D networks are used to extract the frame features and aggregate them to build a rich video-level representation. Results: We validate our model using our private dataset by five-cross validation. For ASD detection, we achieve 89.33 AUC, 84.95 accuracy, 85.70 sensitivity, 81.51 specificity and 81.99 F1 score. Conclusion: The proposed model is multiple instances learning-based deep learning model for video atrial septal defect detection which effectively improves ASD detection accuracy when compared to the performances of previous networks and clinical doctors.

Motivation & Objective

• To develop an automated, deep learning-based system for early and accurate detection of atrial septal defects (ASD) in children using echocardiography videos.
• To address the challenge of limited temporal presence of shunt signals in ultrasound videos by designing a robust video-level representation learning strategy.
• To improve diagnostic accuracy and reduce subjectivity in ASD detection, especially in settings with limited access to experienced echocardiographers.
• To evaluate the model’s performance against clinical experts and demonstrate its potential as a decision-support tool in pediatric cardiology.

Proposed method

• The model uses two standard echocardiographic views—subcostal (subAS) and low parasternal four-chamber (LPS4C)—as input for ASD detection.
• A block random selection strategy is applied during training to enhance model generalization by sampling non-overlapping video segments.
• Frame-level features are extracted using a pre-trained ResNet18 backbone, followed by 3D convolutional network (R3D) for temporal feature aggregation.
• An attention-based aggregation module (AAM) is introduced to emphasize salient regions, particularly color Doppler shunt signals, in the video sequence.
• A maximal agreement decision mechanism is used during inference to improve prediction robustness by aggregating predictions across multiple sampled video blocks.
• The model is trained and validated using five-fold cross-validation on a private dataset of 300 pediatric patients with confirmed ASD status.

Figure 1 : Inclusion and exclusion criteria for the main cohort of this study

Experimental results

Research questions

• RQ1 Can a deep learning model effectively detect ASD in pediatric ultrasound videos despite the transient appearance of shunt signals?
• RQ2 How does the proposed multiple instance learning strategy improve detection performance compared to standard video classification networks?
• RQ3 To what extent does the model outperform clinical experts in ASD detection accuracy when tested on a blinded set of 30 pediatric cases?
• RQ4 What is the contribution of key components—such as attention aggregation, block random sampling, and maximal agreement decision—to overall model performance?

Key findings

• The model achieved 84.95% accuracy, 89.33% AUC, 85.70% sensitivity, 81.51% specificity, and 81.99% F1 score on five-fold cross-validation using a private dataset of 300 children.
• In a blinded clinician comparison, the model achieved 83.33% accuracy, surpassing junior doctors (63.66%) and senior doctors (71.70%).
• The model demonstrated a higher positive predictive value (PPV) of 85.71% compared to doctors, indicating stronger performance in reducing false positives.
• Heatmap visualization confirmed that the model focuses on color Doppler shunt signals, validating its clinical relevance and attention to diagnostic markers.
• Ablation studies showed that combining 3D fusion, attention aggregation, block random sampling, and maximal agreement decision improved accuracy by 7.7% over baseline.
• The model outperformed 2D and 3D baseline networks, with the 2D+3D hybrid architecture proving more effective than pure 3D models for ASD detection.

Figure 2 : The visualization of typical ASD and normal static images from the collected dataset. (A) an ASD frame from the subcostal view of the atrial septum, (B) an ASD frame from the low parasternal four-chamber view, (C) a normal frame from the subcostal view of the atrial septum, (D) a normal f