[Paper Review] Agent with Warm Start and Active Termination for Plane Localization in 3D Ultrasound

Haoran Dou, Xin Yang+9 more|arXiv (Cornell University)|Oct 10, 2019

Fetal and Pediatric Neurological Disorders10 references4 citations

TL;DR

This paper proposes a reinforcement learning (RL) framework with landmark-aware warm start and RNN-based active termination for automatic fetal brain plane localization in 3D ultrasound. By leveraging anatomical atlases for spatial guidance and adaptive stopping via recurrent networks, the method achieves high accuracy (2.7 mm/9.1° for transthalamic plane) and reduces inference steps by 87%, significantly improving efficiency and robustness over baseline methods.

ABSTRACT

Standard plane localization is crucial for ultrasound (US) diagnosis. In prenatal US, dozens of standard planes are manually acquired with a 2D probe. It is time-consuming and operator-dependent. In comparison, 3D US containing multiple standard planes in one shot has the inherent advantages of less user-dependency and more efficiency. However, manual plane localization in US volume is challenging due to the huge search space and large fetal posture variation. In this study, we propose a novel reinforcement learning (RL) framework to automatically localize fetal brain standard planes in 3D US. Our contribution is two-fold. First, we equip the RL framework with a landmark-aware alignment module to provide warm start and strong spatial bounds for the agent actions, thus ensuring its effectiveness. Second, instead of passively and empirically terminating the agent inference, we propose a recurrent neural network based strategy for active termination of the agent's interaction procedure. This improves both the accuracy and efficiency of the localization system. Extensively validated on our in-house large dataset, our approach achieves the accuracy of 3.4mm/9.6° and 2.7mm/9.1° for the transcerebellar and transthalamic plane localization, respectively. Ourproposed RL framework is general and has the potential to improve the efficiency and standardization of US scanning.

Motivation & Objective

• Address the challenge of manual, time-consuming, and operator-dependent standard plane localization in 3D ultrasound for prenatal fetal brain assessment.
• Overcome the limitations of existing methods—such as poor generalization under image noise, artifacts, and fetal posture variation—by leveraging reinforcement learning.
• Improve the effectiveness and efficiency of RL-based plane localization through a landmark-aware alignment module for warm start and an RNN-based active termination strategy.
• Develop a generalizable framework applicable to multiple standard planes in 3D ultrasound volumes with minimal human intervention.

Proposed method

• Employ a deep reinforcement learning agent to iteratively interact with 3D ultrasound volumes, selecting actions to refine plane configuration (normal vector and offset) toward optimal alignment.
• Introduce a landmark-aware alignment module that detects key anatomical landmarks (e.g., corpus callosum, cerebellar vermis) and registers them to a subject-specific atlas, providing strong spatial priors for agent initialization.
• Use a double deep Q-network (DDQN) as the RL backbone to stabilize training and improve policy learning in high-dimensional action spaces.
• Implement a recurrent neural network (RNN) as an active termination module that analyzes the sequence of Q-values during inference to detect the optimal stopping point, avoiding premature or excessive iterations.
• Integrate the RNN-based termination strategy with DDQN to dynamically decide when to halt interaction, balancing accuracy and computational cost.
• Train and validate the framework on a large in-house dataset of 100 3D ultrasound volumes, using metrics including angular deviation (Ang), distance to origin (Dis), and SSIM for evaluation.

Experimental results

Research questions

• RQ1 Can a reinforcement learning agent effectively localize standard fetal brain planes in 3D ultrasound volumes despite high image noise and large anatomical variability?
• RQ2 How does incorporating anatomical landmark detection and atlas-based alignment improve the convergence and accuracy of RL-based plane localization?
• RQ3 Can an RNN-based active termination mechanism outperform fixed or heuristic stopping criteria in terms of both localization accuracy and inference efficiency?
• RQ4 To what extent does the proposed framework reduce the number of inference steps compared to standard RL agents with fixed termination policies?
• RQ5 Does the integration of prior anatomical knowledge through landmark alignment lead to measurable improvements in localization performance across diverse fetal postures?

Key findings

• The proposed method achieved a localization accuracy of 3.4 mm/9.6° for the transcerebellar plane and 2.7 mm/9.1° for the transthalamic plane, significantly outperforming baseline regression and registration-based methods.
• The landmark-aware alignment module reduced angular error by over 50% compared to the baseline DDQN without alignment (DDQN-nA), demonstrating its effectiveness in providing strong spatial priors.
• The RNN-based active termination strategy reduced the average number of inference steps from 100 to 13, achieving a 87% reduction in computational cost while improving accuracy.
• Among all variants, DDQN-AT (LSTM) achieved the best performance with 9.11°/2.66 mm for TT and 9.53°/2.64 mm for TC, indicating superior sequence modeling capability.
• The active termination strategy enabled earlier convergence without sacrificing accuracy, as evidenced by the optimal termination step closely matching the ground truth in visualizations.
• The framework demonstrated robustness across diverse fetal postures and image qualities, with consistent performance across 100 test volumes, validating its generalization potential.