[Paper Review] Deep learning with spatiotemporal consistency for nerve segmentation in ultrasound images

Adel Hafiane, Pierre Vieyres+1 more|arXiv (Cornell University)|Jun 19, 2017

Anesthesia and Pain Management17 citations

TL;DR

This paper proposes a deep learning framework with spatiotemporal consistency to improve nerve segmentation in ultrasound images for ultrasound-guided regional anesthesia. By combining a CNN for initial nerve localization with spatial and temporal coherence to reduce false positives, followed by probabilistic active contours for precise boundary delineation, the method achieves a 96% F-score in nerve localization and high segmentation accuracy.

ABSTRACT

Ultrasound-Guided Regional Anesthesia (UGRA) has been gaining importance in the last few years, offering numerous advantages over alternative methods of nerve localization (neurostimulation or paraesthesia). However, nerve detection is one of the most tasks that anaesthetists can encounter in the UGRA procedure. Computer aided system that can detect automatically region of nerve, would help practitioner to concentrate more in anaesthetic delivery. In this paper we propose a new method based on deep learning combined with spatiotemporal information to robustly segment the nerve region. The proposed method is based on two phases, localisation and segmentation. The first phase, consists in using convolutional neural network combined with spatial and temporal consistency to detect the nerve zone. The second phase utilises active contour model to delineate the region of interest. Obtained results show the validity of the proposed approach and its robustness.

Motivation & Objective

• Address the challenge of unreliable nerve detection in ultrasound-guided regional anesthesia (UGRA), where manual localization is time-consuming and requires expert skill.
• Reduce false positive detections in nerve localization caused by noise and artifacts in ultrasound images.
• Leverage dynamic scanning behavior—spatial and temporal consistency—inspired by how human experts use probe motion to improve confidence in nerve identification.
• Improve segmentation accuracy by using the refined localization as an initial contour for active contour models.
• Demonstrate that integrating spatiotemporal coherence with deep learning outperforms traditional methods like SVM and standalone CNNs.

Proposed method

• Use a 3-convolutional-layer CNN with ReLU activation and max-pooling to classify image patches (64×64) as nerve or non-nerve regions.
• Apply spatial coherence by retaining only ROIs that show consistent detection across multiple frames in a sequence.
• Enforce temporal consistency by requiring a detected ROI to remain stable across consecutive ultrasound frames during steady probe movement.
• Use a soft voting mechanism based on intersection-over-union (IoU) across frames to filter out noisy or transient detections.
• Feed the refined ROI as initial contour into a probabilistic geodesic active contour (PGVF) model to accurately segment nerve boundaries.
• Optimize the CNN using stochastic gradient descent with 0.5 dropout on the fully connected layer for regularization.

Experimental results

Research questions

• RQ1 Can spatiotemporal consistency significantly reduce false positive detections in nerve localization from ultrasound images?
• RQ2 Does combining CNN-based feature learning with temporal and spatial coherence improve localization robustness compared to standalone CNN or SVM?
• RQ3 To what extent does using dynamic scanning information (probe motion) enhance detection reliability in a clinical setting?
• RQ4 Can the integration of deep learning with active contour models yield superior segmentation accuracy when initialized with consistent ROIs?
• RQ5 How does the proposed method compare quantitatively to existing approaches like SVM with feature selection or temporal constraints?

Key findings

• The proposed method achieved an F-score of 0.96 ± 0.25 in nerve localization, outperforming standalone CNN (0.91 ± 0.24) and SVM with feature selection (0.88 ± 0.52).
• Incorporating spatiotemporal consistency improved the F-score by 5% compared to CNN alone, indicating effective suppression of false positives.
• The method achieved a Dice coefficient of 0.85 ± 0.15 and a Hausdorff distance of 10.72 ± 3.91 for segmentation, showing high overlap with expert annotations.
• The combination of CNN with spatiotemporal constraints reduced the false positive rate more effectively than SVM with temporal constraints (F-score: 0.92 ± 0.33).
• Qualitative results showed that the segmentation closely matched expert-drawn ground truth, with accurate delineation even in low-contrast or noisy regions.
• The framework demonstrated robustness across ten patient videos with an average of 500 frames per video, validating its performance on real-world dynamic ultrasound sequences.