[Paper Review] Image-Guided Navigation of a Robotic Ultrasound Probe for Autonomous Spinal Sonography Using a Shadow-aware Dual-Agent Framework

Keyu Li, Yangxin Xu+4 more|arXiv (Cornell University)|Nov 3, 2021

Soft Robotics and Applications43 references45 citations

TL;DR

This paper proposes a shadow-aware dual-agent framework that combines reinforcement learning (RL) and deep learning (DL) to autonomously navigate a robotic ultrasound probe for standard spinal sonography views. The RL agent plans 6-DOF probe motion using real-time US images, while a DL agent detects standard views; a novel view-specific acoustic shadow reward guides navigation, achieving 5.18 mm/5.25° accuracy in intra-subject and 12.87 mm/17.49° in inter-subject settings.

ABSTRACT

Ultrasound (US) imaging is commonly used to assist in the diagnosis and interventions of spine diseases, while the standardized US acquisitions performed by manually operating the probe require substantial experience and training of sonographers. In this work, we propose a novel dual-agent framework that integrates a reinforcement learning (RL) agent and a deep learning (DL) agent to jointly determine the movement of the US probe based on the real-time US images, in order to mimic the decision-making process of an expert sonographer to achieve autonomous standard view acquisitions in spinal sonography. Moreover, inspired by the nature of US propagation and the characteristics of the spinal anatomy, we introduce a view-specific acoustic shadow reward to utilize the shadow information to implicitly guide the navigation of the probe toward different standard views of the spine. Our method is validated in both quantitative and qualitative experiments in a simulation environment built with US data acquired from 17 volunteers. The average navigation accuracy toward different standard views achieves 5.18mm/5.25deg and 12.87mm/17.49deg in the intra- and inter-subject settings, respectively. The results demonstrate that our method can effectively interpret the US images and navigate the probe to acquire multiple standard views of the spine.

Motivation & Objective

• To reduce the cognitive and physical burden on sonographers by enabling autonomous acquisition of standard spinal ultrasound views.
• To address the challenge of robotic probe navigation in medical ultrasound, which requires real-time image interpretation and precise motion planning.
• To improve navigation accuracy and robustness in the presence of anatomical variability across patients.
• To develop a simulation environment that realistically models probe-tissue interactions using real US data for training and validation.
• To integrate acoustic shadow information—often ignored—as a task-specific reward signal to implicitly guide probe navigation toward target spinal views.

Proposed method

• A deep reinforcement learning (RL) agent is trained end-to-end to control the 6-DOF movement of a robotic ultrasound probe based on real-time B-mode US images.
• A view-specific acoustic shadow reward (ASR) is introduced to leverage the natural acoustic shadowing in spinal US images as implicit guidance for RL navigation.
• A pre-trained deep learning (DL) agent performs real-time standard view recognition (PSL, PSAP, TSP) from US images to provide feedback and jointly determine probe motion.
• The dual-agent framework uses a collaborative navigation workflow where the RL agent receives navigation rewards and the DL agent provides view recognition feedback under safety constraints.
• A simulation environment is built using real US data from 17 volunteers, enabling continuous state and action spaces for training and evaluation.
• The RL agent uses a stacked observation of the 4 most recent US frames to incorporate temporal dynamics into decision-making.

Experimental results

Research questions

• RQ1 Can a dual-agent RL-DL framework effectively guide a robotic ultrasound probe to autonomously acquire standard spinal views using only real-time US images?
• RQ2 How does incorporating view-specific acoustic shadow information improve navigation accuracy and robustness in spinal sonography?
• RQ3 To what extent can the proposed framework generalize across different subjects with anatomical variability?
• RQ4 How does the collaboration between the RL and DL agents enhance navigation performance compared to a standalone RL agent?
• RQ5 Can a simulation environment based on real US data effectively support the training and validation of robotic US navigation policies?

Key findings

• The proposed framework achieves an average navigation accuracy of 5.18 mm and 5.25° in the intra-subject setting, demonstrating high precision in controlled conditions.
• In the more challenging inter-subject setting, the average navigation accuracy is 12.87 mm and 17.49°, indicating robustness despite anatomical variability.
• The integration of the view-specific acoustic shadow reward significantly improves navigation performance compared to baseline RL agents without this signal.
• The dual-agent collaborative navigation workflow enhances both accuracy and efficiency, as shown by the improved convergence and reduced pose error in final stopping positions.
• The simulation environment successfully reproduces real-world US acquisition dynamics, enabling effective training and evaluation of the navigation policy.
• The method shows potential for clinical deployment, though generalization across diverse patient anatomies remains a challenge requiring larger datasets.