[Paper Review] Fully Automatic Segmentation of 3D Brain Ultrasound: Learning from Coarse Annotations

Julia Rackerseder, Rüdiger Göbl+2 more|arXiv (Cornell University)|Apr 18, 2019

Medical Image Segmentation Techniques22 references4 citations

TL;DR

This paper proposes a fully automatic 3D brain ultrasound segmentation method using DenseVNet with pre-training on simulated ultrasound sweeps, enabling robust initialization for image registration despite training on coarse and incomplete semi-automatic annotations. The approach achieves a Dice score of 0.47±0.10 and average surface distance of 0.71±0.38 mm, outperforming baseline models and producing more complete, generalizable segmentations than the input annotations.

ABSTRACT

Intra-operative ultrasound is an increasingly important imaging modality in neurosurgery. However, manual interaction with imaging data during the procedures, for example to select landmarks or perform segmentation, is difficult and can be time consuming. Yet, as registration to other imaging modalities is required in most cases, some annotation is necessary. We propose a segmentation method based on DeepVNet and specifically evaluate the integration of pre-training with simulated ultrasound sweeps to improve automatic segmentation and enable a fully automatic initialization of registration. In this view, we show that despite training on coarse and incomplete semi-automatic annotations, our approach is able to capture the desired superficial structures such as extit{sulci}, the extit{cerebellar tentorium}, and the extit{falx cerebri}. We perform a five-fold cross-validation on the publicly available RESECT dataset. Trained on the dataset alone, we report a Dice and Jaccard coefficient of $0.45 \pm 0.09$ and $0.30 \pm 0.07$ respectively, as well as an average distance of $0.78 \pm 0.36~mm$. With the suggested pre-training, we computed a Dice and Jaccard coefficient of $0.47 \pm 0.10$ and $0.31 \pm 0.08$, and an average distance of $0.71 \pm 0.38~mm$. The qualitative evaluation suggest that with pre-training the network can learn to generalize better and provide refined and more complete segmentations in comparison to incomplete annotations provided as input.

Motivation & Objective

• To enable fully automatic segmentation of 3D brain ultrasound for intra-operative image registration, reducing reliance on manual annotation.
• To address the challenge of limited, coarse, and incomplete ground-truth annotations in 3D ultrasound by leveraging simulation-based pre-training.
• To improve the robustness and generalization of segmentation models in the presence of brain shift and poor-quality input data.
• To develop a method that produces more complete and refined segmentations than the input coarse annotations, suitable for registration initialization.
• To demonstrate that even with small datasets (23 patients) and low-quality labels, deep learning can yield clinically useful segmentation outputs.

Proposed method

• A fully convolutional DenseVNet architecture is used for 3D brain ultrasound segmentation, enabling end-to-end learning from volumetric ultrasound data.
• Pre-training is performed on synthetic ultrasound sweeps generated via simulation to improve feature learning and generalization before fine-tuning on real data.
• The model is trained exclusively on coarse and incomplete semi-automatic segmentations, which are derived from a prior method but contain errors and missing regions.
• Five-fold cross-validation is applied on the publicly available RESECT dataset to evaluate performance robustly across subjects.
• Evaluation metrics include Dice coefficient, Jaccard index, average surface distance, and Hausdorff distance to assess segmentation accuracy and completeness.
• Qualitative analysis compares predicted segmentations against manual ground truth and coarse input annotations to assess refinement and structural coverage.

Experimental results

Research questions

• RQ1 Can a deep learning model achieve fully automatic 3D brain ultrasound segmentation when trained only on coarse and incomplete semi-automatic annotations?
• RQ2 Does pre-training on simulated ultrasound sweeps improve the generalization and accuracy of 3D US segmentation models in the absence of high-quality ground truth?
• RQ3 Can the resulting segmentation be more complete and refined than the input coarse annotations, even when the input is patchy or erroneous?
• RQ4 To what extent does the model generalize across subjects, especially in cases with significant brain shift or poor image quality?
• RQ5 Is the proposed method suitable for initializing image registration in neurosurgical interventions, given the clinical need for robust and automated initialization?

Key findings

• The proposed method achieves a Dice coefficient of 0.47±0.10 and Jaccard index of 0.31±0.08 with pre-training, representing a statistically significant improvement over the baseline (0.45±0.09 and 0.30±0.07).
• The average surface distance is reduced to 0.71±0.38 mm with pre-training, indicating improved geometric accuracy compared to 0.78±0.36 mm without it.
• Despite low Dice scores, the predicted segmentations are more complete and cover more of the target structures (e.g., sulci, falx cerebri, cerebellar tentorium) than the coarse input annotations.
• The model generalizes well across subjects, with qualitative results showing refined and coherent segmentations even when input labels are patchy or erroneous.
• Patients 1 and 21 are outliers due to poor input annotations, but the model still produces segmentations much closer to the manual ground truth than the input labels.
• The Hausdorff distance remains high (20.70±5.64 mm) in both settings, indicating residual outliers, but the average distance is low, suggesting overall good alignment.