[Paper Review] Toward Personalized Training and Skill Assessment in Robotic Minimally Invasive Surgery

Mahtab J. Fard, Sattar Ameri+1 more|arXiv (Cornell University)|Oct 23, 2016

Surgical Simulation and Training10 references3 citations

TL;DR

This paper proposes a shape-based, dynamic time warping (DTW)-enabled k-NN framework for automated, personalized skill assessment and real-time feedback in robotic minimally invasive surgery (RMIS). By directly analyzing raw kinematic time series data without extensive preprocessing, the method achieves 82% accuracy in suturing, 70% in needle passing, and 85% in knot tying—outperforming or matching state-of-the-art approaches with minimal parameter tuning and enabling online performance feedback for surgeons during training.

ABSTRACT

Despite the immense technology advancement in the surgeries the criteria of assessing the surgical skills still remains based on subjective standards. With the advent of robotic-assisted surgery, new opportunities for objective and autonomous skill assessment is introduced. Previous works in this area are mostly based on structured-based method such as Hidden Markov Model (HMM) which need enormous pre-processing. In this study, in contrast with them, we develop a new shaped-based framework for automatically skill assessment and personalized surgical training with minimum parameter tuning. Our work has addressed main aspects of skill evaluation; develop gesture recognition model directly on temporal kinematic signal of robotic-assisted surgery, and build automated personalized RMIS gesture training framework which . We showed that our method, with an average accuracy of 82% for suturing, 70% for needle passing and 85% for knot tying, performs better or equal than the state-of-the-art methods, while simultaneously needs minimum pre-processing, parameter tuning and provides surgeons with online feedback for their performance during training.

Motivation & Objective

• To develop an objective, automated, and personalized framework for surgical skill assessment in robotic minimally invasive surgery (RMIS) to overcome the limitations of subjective, expert-based evaluation.
• To enable real-time, online feedback during surgical training by identifying performance deviations from expert benchmarks using time series similarity.
• To reduce reliance on complex preprocessing and extensive parameter tuning by directly classifying raw kinematic signals using shape-based similarity measures.
• To support personalized training by identifying specific surgical gestures that require improvement for individual surgeons.
• To lay the foundation for semi-autonomous robotic behaviors, such as automatic camera control, through real-time gesture recognition.

Proposed method

• The method uses Dynamic Time Warping (DTW) to compute similarity between temporal kinematic signals of surgical gestures, enabling robust comparison of time series with variable timing.
• DTW distances are used as input to a k-Nearest Neighbors (k-NN) classifier to assign new gestures to predefined surgical task categories.
• The framework operates directly on raw kinematic data, avoiding feature extraction and reducing preprocessing complexity.
• A confusion matrix heatmap is generated to compare novice and expert surgeons, identifying specific gestures with performance discrepancies.
• The system provides online augmented feedback by detecting when a surgeon's gesture deviates from the expert pattern.
• The approach relies on a predefined surgical gesture dictionary (Table 2) and expert-annotated ground truth labels for training.

Experimental results

Research questions

• RQ1 Can a shape-based time series classification method achieve high accuracy in surgical gesture recognition using only raw kinematic data from RMIS?
• RQ2 Can such a method provide real-time, personalized feedback during surgical training by comparing novice performance to expert benchmarks?
• RQ3 How does the performance of this DTW-kNN approach compare to existing HMM- and feature-based methods in terms of accuracy and computational simplicity?
• RQ4 What are the key gestures that novice surgeons perform incorrectly compared to experts, and can these be systematically identified?
• RQ5 To what extent can this framework support the development of semi-autonomous robotic systems in surgery?

Key findings

• The proposed framework achieved an average accuracy of 82% in suturing, 70% in needle passing, and 85% in knot tying, demonstrating strong performance on real RMIS tasks.
• The method outperformed or matched state-of-the-art approaches while requiring minimal preprocessing and parameter tuning.
• Confusion matrix heatmaps revealed that novice surgeons required more training for specific gestures—e.g., G4, G6, and G3 in suturing, and most gestures in needle passing.
• Knot tying was the most straightforward task, with only G12 requiring notable retraining for novices.
• The results suggest that the current global surgical gesture dictionary may need revision to support a universal surgical language.
• The study highlights the critical role of accurate ground truth labeling, as classification performance heavily depends on the quality of expert-annotated labels.