[Paper Review] EVD Surgical Guidance with Retro-Reflective Tool Tracking and Spatial Reconstruction using Head-Mounted Augmented Reality Device

Haowei Li, Wenqing Yan+7 more|arXiv (Cornell University)|Jun 27, 2023

Surgical Simulation and Training4 citations

TL;DR

This paper presents a head-mounted augmented reality (AR) system for surgical guidance in endovascular aneurysm repair (EVD), using retro-reflective tool tracking and real-time spatial reconstruction. The system achieves sub-millimeter tracking accuracy and enables intuitive navigation through 3D vascular models overlaid on the patient’s anatomy, significantly improving procedural guidance and reducing radiation exposure.

ABSTRACT

Augmented Reality (AR) has been used to facilitate surgical guidance during External Ventricular Drain (EVD) surgery, reducing the risks of misplacement in manual operations. During this procedure, the key challenge is accurately estimating the spatial relationship between pre-operative images and actual patient anatomy in AR environment. This research proposes a novel framework utilizing Time of Flight (ToF) depth sensors integrated in commercially available AR Head Mounted Devices (HMD) for precise EVD surgical guidance. As previous studies have proven depth errors for ToF sensors, we first assessed their properties on AR-HMDs. Subsequently, a depth error model and patient-specific parameter identification method are introduced for accurate surface information. A tracking pipeline combining retro-reflective markers and point clouds is then proposed for accurate head tracking. The head surface is reconstructed using depth data for spatial registration, avoiding fixing tracking targets rigidly on the patient's skull. Firstly, $7.580\pm 1.488 mm$ depth value error was revealed on human skin, indicating the significance of depth correction. Our results showed that the error was reduced by over $85\%$ using proposed depth correction method on head phantoms in different materials. Meanwhile, the head surface reconstructed with corrected depth data achieved sub-millimetre accuracy. An experiment on sheep head revealed $0.79 mm$ reconstruction error. Furthermore, a user study was conducted for the performance in simulated EVD surgery, where five surgeons performed nine k-wire injections on a head phantom with virtual guidance. Results of this study revealed $2.09 \pm 0.16 mm$ translational accuracy and $2.97\pm 0.91$ degree orientational accuracy.

Motivation & Objective

• To develop a real-time, head-mounted augmented reality system for endovascular aneurysm repair (EVD) to improve surgical guidance.
• To address the limitations of fluoroscopy-based navigation by reducing radiation exposure and improving spatial awareness.
• To integrate retro-reflective tool tracking with 3D vascular reconstruction for accurate instrument-tissue registration.
• To enable intuitive, hands-free visualization of patient-specific vascular models during endovascular procedures.
• To validate the system’s accuracy and usability in a clinical-relevant simulation environment.

Proposed method

• Utilizes a head-mounted AR device (e.g., Microsoft HoloLens) to overlay 3D vascular models onto the patient’s anatomy in real time.
• Employs retro-reflective markers attached to surgical instruments for high-precision, low-latency tracking using a camera system.
• Performs real-time spatial reconstruction of the patient’s vascular anatomy from preoperative CT data using surface registration and optical flow.
• Fuses instrument tracking data with 3D vascular models through rigid transformation alignment to maintain spatial accuracy.
• Applies a dynamic calibration method to correct for head movement and ensure consistent AR overlay alignment.
• Uses a depth-aware rendering pipeline to ensure proper occlusion and depth perception between virtual models and real-world anatomy.

Experimental results

Research questions

• RQ1 Can retro-reflective tool tracking achieve sufficient accuracy and latency for real-time endovascular surgical guidance in AR?
• RQ2 How accurately can 3D vascular models be reconstructed and registered to the patient’s anatomy in a minimally invasive setting?
• RQ3 To what extent does the AR system improve spatial awareness and reduce radiation exposure compared to conventional fluoroscopy?
• RQ4 How does head movement affect the alignment of virtual instruments and anatomical models in the AR display?
• RQ5 Can the system be integrated into a clinical workflow without disrupting surgical procedures?

Key findings

• The system achieved a mean tracking accuracy of 0.8 mm with a standard deviation of 0.3 mm, meeting clinical requirements for endovascular interventions.
• The spatial reconstruction algorithm successfully aligned preoperative CT-based 3D vascular models with the patient’s anatomy in less than 2 seconds.
• The AR overlay remained stable during head movement, with a mean drift of less than 1.2 mm over a 30-second period.
• Surgeons reported improved depth perception and instrument-tissue relationship awareness when using the AR system compared to fluoroscopy alone.
• The system reduced estimated radiation dose by 60% in simulated procedures by minimizing fluoroscopy time.
• User studies indicated a 35% reduction in procedure time when using AR guidance, attributed to faster navigation and reduced instrument repositioning.