[Paper Review] Real-time Prediction of Soft Tissue Deformations Using Data-driven Nonlinear Presurgical Simulations

Haolin Liu, Ye Han+5 more|arXiv (Cornell University)|Oct 26, 2020

Medical Imaging and Analysis25 references4 citations

TL;DR

This paper proposes a data-driven artificial neural network (ANN) approach for real-time prediction of soft tissue deformations in image-guided surgery using sparsely registered fiducial markers. By leveraging finite element method (FEM)-generated deformation benchmarks and principal component analysis (PCA) for dimensionality reduction, the method achieves sub-millimeter accuracy (mean offset <1.0 mm for 98% of cases) with prediction times under 0.5 seconds, demonstrating strong clinical potential for intraoperative soft tissue tracking.

ABSTRACT

Imaging modalities provide clinicians with real-time visualization of anatomical regions of interest (ROI) for the purpose of minimally invasive surgery. During the procedure, low-resolution image data are acquired and registered with high-resolution preoperative 3D reconstruction to guide the execution of surgical preplan. Unfortunately, due to the potential large strain and nonlinearities in the deformation of soft biological tissues, significant mismatch may be observed between ROI shapes during pre- and intra-operative imaging stages, making the surgical preplan prone to failure. In an effort to bridge the gap between the two imaging stages, this paper presents a data-driven approach based on artificial neural network for predicting the ROI deformation in real-time with sparsely registered fiducial markers. For a head-and-neck tumor model with an average maximum displacement of 30 mm, the maximum surface offsets between benchmarks and predictions using the proposed approach for 98% of the test cases are under 1.0 mm, which is the typical resolution of high-quality interventional ultrasound. Each of the prediction processes takes less than 0.5 s. With the resulting prediction accuracy and computational efficiency, the proposed approach demonstrates its potential to be clinically relevant.

Motivation & Objective

• To address the challenge of soft tissue deformation mismatch between preoperative 3D reconstructions and intraoperative imaging in image-guided surgery.
• To enable real-time, accurate deformation prediction despite sparse intraoperative fiducial marker data and large, nonlinear tissue strains.
• To overcome the computational infeasibility of repeated finite element method (FEM) simulations during surgery by replacing them with a fast, learned model.
• To develop a clinically viable, patient-specific framework that maintains high accuracy across diverse deformation scenarios.
• To demonstrate the feasibility of using artificial neural networks trained on FEM-generated benchmarks for real-time soft tissue reconstruction.

Proposed method

• The method uses preoperative 3D patient-specific models of head-and-neck tumors and selects mesh nodes as fiducial markers (FMs).
• A large dataset of nonlinear FEM simulations is generated, tracking displacements of the selected FMs under various loading conditions.
• Principal component analysis (PCA) is applied to the displacement fields to extract low-dimensional representations (principal components) of deformation modes.
• An artificial neural network (ANN) is trained on the PCA coefficients and corresponding FM displacements to learn the mapping from sparse FM data to full deformation fields.
• During surgery, only the intraoperative FM positions are input into the trained ANN, which predicts the full deformed soft tissue configuration in real time.
• The approach combines physics-based FEM simulations for data generation with machine learning for fast inference, enabling real-time performance.

Experimental results

Research questions

• RQ1 Can a data-driven ANN model achieve real-time, sub-millimeter accurate prediction of soft tissue deformations using only sparse fiducial marker data?
• RQ2 How does the performance of the proposed ANN-based method compare to traditional ridge regression in terms of accuracy and speed?
• RQ3 To what extent can the trained ANN generalize to unseen deformation scenarios with different force fields?
• RQ4 What is the optimal architecture of the ANN in terms of hidden layer size and number of principal components for balancing accuracy and inference speed?
• RQ5 Can the method maintain high accuracy and low latency across complex, large-strain deformations typical in head-and-neck surgical scenarios?

Key findings

• For 98% of test cases, the maximum surface offset between the predicted and benchmark deformed configurations was under 1.0 mm, matching the resolution of high-quality interventional ultrasound.
• The average prediction time across all test cases was less than 0.5 seconds, demonstrating strong real-time performance.
• The proposed ANN method outperformed ridge regression in both accuracy and speed, with significantly lower mean and maximum offsets.
• The optimal ANN architecture was found to be [128, 64] hidden layers, achieving the best trade-off between reconstruction accuracy and training time.
• The model generalized well to unseen deformation scenarios generated with a different force field strategy, confirming robustness and generalization capability.
• The method maintained high accuracy (1.5% error relative to tumor’s maximum dimension) even on complex, large-deformation cases with up to 30 mm displacement.