[Paper Review] Pubic Symphysis-Fetal Head Segmentation Using Pure Transformer with Bi-level Routing Attention

Pengzhou Cai, Jiang, Lu+2 more|arXiv (Cornell University)|Sep 30, 2023

Cleft Lip and Palate Research4 citations

TL;DR

This paper proposes BRAU-Net, a pure Transformer-based U-Net architecture with bi-level routing attention (BRA) for accurate pubic symphysis-fetal head segmentation in transperineal ultrasound images. The method leverages coarse-to-fine attention routing to enhance long-range dependency modeling and achieves a final score of 89.74 on the FH-PS-AOP challenge test set, demonstrating strong performance in clinical fetal head station assessment.

ABSTRACT

In this paper, we propose a method, named BRAU-Net, to solve the pubic symphysis-fetal head segmentation task. The method adopts a U-Net-like pure Transformer architecture with bi-level routing attention and skip connections, which effectively learns local-global semantic information. The proposed BRAU-Net was evaluated on transperineal Ultrasound images dataset from the pubic symphysis-fetal head segmentation and angle of progression (FH-PS-AOP) challenge. The results demonstrate that the proposed BRAU-Net achieves comparable a final score. The codes will be available at https://github.com/Caipengzhou/BRAU-Net.

Motivation & Objective

• To develop a robust, fully Transformer-based segmentation model for pubic symphysis and fetal head in transperineal ultrasound images.
• To address the limitations of manual segmentation, which is time-consuming and suffers from high inter-observer variability.
• To improve objective, accurate, and reproducible assessment of fetal head descent via automated segmentation and angle of progression (AoP) measurement.
• To integrate bi-level routing attention into a U-Net-like architecture to effectively capture both local and global semantic features.
• To achieve state-of-the-art performance on the pubic symphysis-fetal head segmentation and angle of progression (FH-PS-AOP) challenge.

Proposed method

• The model employs a U-Net-like architecture with a pure Transformer encoder-decoder backbone, replacing convolutional layers with self-attention mechanisms.
• The core component is the BiFormer block, which applies bi-level routing attention to dynamically route attention between spatial regions and within selected regions.
• Bi-level routing involves region-level attention computation via query and key pooling over non-overlapping patches, followed by top-k selection of most relevant regions.
• Token-to-token attention is then applied only within the top-k routed regions, reducing computational cost while preserving long-range dependencies.
• A learnable local context encoder (LCE) using a 5x5 depth-wise convolution is added to enhance local feature modeling in the attention mechanism.
• Skip connections are used to fuse multi-scale features from the encoder with the decoder, preserving spatial resolution and improving localization accuracy.

Experimental results

Research questions

• RQ1 Can a pure Transformer-based U-Net with bi-level routing attention achieve competitive performance in pubic symphysis-fetal head segmentation on transperineal ultrasound images?
• RQ2 How does bi-level routing attention improve the modeling of long-range dependencies while maintaining computational efficiency in medical image segmentation?
• RQ3 To what extent does the proposed BRAU-Net reduce inter-observer variability compared to manual segmentation in fetal head station assessment?
• RQ4 How well does the model generalize to the FH-PS-AOP challenge test set, particularly in terms of Dice score, Hausdorff distance, and AoP estimation accuracy?
• RQ5 Can the integration of local context encoding (LCE) with attention mechanisms further enhance segmentation precision in low-contrast ultrasound images?

Key findings

• BRAU-Net achieved a final score of 89.74 on the FH-PS-AOP challenge test set, demonstrating strong generalization and clinical relevance.
• The model achieved a Dice-Similarity Coefficient (DSC) of 0.88 for the fetal head (FH), 0.80 for the pubic symphysis (PS), and 0.87 for the combined region (ALL).
• The Hausdorff Distance (HD) was 20.03 mm for the fetal head and 14.07 mm for the pubic symphysis, indicating good boundary localization.
• The Average Surface Distance (ASD) was 7.10 mm for the fetal head and 4.21 mm for the pubic symphysis, reflecting high segmentation accuracy.
• The estimated Angle of Progression (AOP) was 12.20°, which is consistent with clinical expectations and supports the model’s utility in obstetric decision-making.
• The model achieved a balanced performance across all metrics, with the final score combining DSC, HD, ASD, and AOP, indicating robustness in clinical evaluation.