[Paper Review] Recurrent U-net for automatic pelvic floor muscle segmentation on 3D ultrasound

Frieda van den Noort, Beril Sırmaçek+1 more|arXiv (Cornell University)|Jul 29, 2021

Pelvic floor disorders treatments48 references4 citations

TL;DR

This paper proposes a Recurrent U-Net (RU-Net) architecture that integrates Convolutional Long Short-Term Memory (CLSTM) units into a 2D U-Net to enable 3D context learning for automatic segmentation of the levator ani muscle (LAM) in 3D transperineal ultrasound (TPUS) volumes. The method achieves segmentation performance on par with intra-observer manual segmentation, demonstrating significant improvement over standard 2D U-Net on 3D TPUS data.

ABSTRACT

The prevalance of pelvic floor problems is high within the female population. Transperineal ultrasound (TPUS) is the main imaging modality used to investigate these problems. Automating the analysis of TPUS data will help in growing our understanding of pelvic floor related problems. In this study we present a U-net like neural network with some convolutional long short term memory (CLSTM) layers to automate the 3D segmentation of the levator ani muscle (LAM) in TPUS volumes. The CLSTM layers are added to preserve the inter-slice 3D information. We reach human level performance on this segmentation task. Therefore, we conclude that we successfully automated the segmentation of the LAM on 3D TPUS data. This paves the way towards automatic in-vivo analysis of the LAM mechanics in the context of large study populations.

Motivation & Objective

• To address the challenge of accurate 3D segmentation of the levator ani muscle (LAM) in 3D transperineal ultrasound (TPUS) volumes, which is critical for studying pelvic floor dysfunction.
• To overcome the limitations of 2D U-Net in capturing inter-slice contextual information essential for 3D structure segmentation in noisy, low-contrast ultrasound data.
• To develop a deep learning architecture that leverages temporal-like recurrence across slices to improve segmentation robustness and accuracy in 3D TPUS data.
• To achieve segmentation performance comparable to human intra-observer variability, enabling reliable large-scale clinical studies on LAM mechanics.

Proposed method

• Adapted the 2D U-Net architecture by replacing selected convolutional layers with Convolutional Long Short-Term Memory (CLSTM) units to model sequential dependencies across slices in 3D TPUS volumes.
• Integrated CLSTM layers in both the encoder and decoder paths of the U-Net at multiple spatial resolution levels to preserve and propagate 3D contextual features.
• Trained the network end-to-end using 3D TPUS volumes with manually segmented LAM annotations, using the Adam optimizer with default hyperparameters.
• Fed slices in a specific order (top-to-bottom for axial views) to improve training stability, avoiding interference from anatomical curvatures that hinder learning.
• Explored transfer learning by retraining from one orientation (e.g., axial) to another (e.g., sagittal), observing faster convergence but higher risk of overfitting.
• Avoided data sampling for class imbalance, as the model converged well without it, though double training on muscle-containing slices slightly accelerated convergence.

Experimental results

Research questions

• RQ1 Can a 2D U-Net architecture enhanced with CLSTM layers effectively learn 3D contextual information from 2D slices in 3D TPUS data for LAM segmentation?
• RQ2 Does the proposed Recurrent U-Net (RU-Net) achieve segmentation performance comparable to human intra-observer variability on 3D TPUS volumes?
• RQ3 How does the inclusion of CLSTM units improve segmentation accuracy over standard 2D U-Net when applied to 3D ultrasound data with high noise and low contrast?
• RQ4 What is the impact of slice ordering and training strategy (e.g., transfer learning) on the convergence and generalization of the RU-Net model?
• RQ5 Can the RU-Net model generalize across different imaging orientations (axial, coronal, sagittal), and does one orientation yield superior results?

Key findings

• The proposed Recurrent U-Net (RU-Net) achieved segmentation performance on par with intra-observer manual segmentation, indicating high reliability and clinical relevance.
• The inclusion of CLSTM layers significantly improved segmentation accuracy over standard 2D U-Net by enabling the model to learn inter-slice contextual dependencies in 3D TPUS data.
• The network demonstrated robust performance across different imaging orientations, with the sagittal view yielding the best results, suggesting it contains more discriminative 3D information.
• Training was sensitive to learning rate, with small changes causing training failure or poor convergence, highlighting the need for careful hyperparameter tuning.
• Retraining from one orientation to another accelerated convergence but increased the risk of overfitting due to the stochastic nature of training, indicating trade-offs in transfer learning.
• No data sampling was required to address class imbalance, as the model achieved good performance without it, though double training on muscle-containing slices slightly improved training speed.