[論文レビュー] TriSAM: Tri-Plane SAM for zero-shot cortical blood vessel segmentation in VEM images

Jia Wan, Wanhua Li+7 more|arXiv (Cornell University)|Jan 25, 2024

Advanced Electron Microscopy Techniques and Applications被引用数 6

ひとこと要約

TriSAM は Tri-Plane トラッキングフレームワークを介して SAM を引き上げ、VEM 画像のゼロショット 3D 血管セグメンテーション手法を導入し、皮質血管の大規模公開ベンチマークとして BvEM を提供します。モデル学習なしでマウス・マカク・人間のデータセットで最先端のゼロショット性能を達成します。

ABSTRACT

While imaging techniques at macro and mesoscales have garnered substantial attention and resources, microscale Volume Electron Microscopy (vEM) imaging, capable of revealing intricate vascular details, has lacked the necessary benchmarking infrastructure. In this paper, we address a significant gap in this field of neuroimaging by introducing the first-in-class public benchmark, BvEM, designed specifically for cortical blood vessel segmentation in vEM images. Our BvEM benchmark is based on vEM image volumes from three mammals: adult mouse, macaque, and human. We standardized the resolution, addressed imaging variations, and meticulously annotated blood vessels through semi-automatic, manual, and quality control processes, ensuring high-quality 3D segmentation. Furthermore, we developed a zero-shot cortical blood vessel segmentation method named TriSAM, which leverages the powerful segmentation model SAM for 3D segmentation. To extend SAM from 2D to 3D volume segmentation, TriSAM employs a multi-seed tracking framework, leveraging the reliability of certain image planes for tracking while using others to identify potential turning points. This approach effectively achieves long-term 3D blood vessel segmentation without model training or fine-tuning. Experimental results show that TriSAM achieved superior performances on the BvEM benchmark across three species. Our dataset, code, and model are available online at \url{https://jia-wan.github.io/bvem}.

研究の動機と目的

• Curate and standardize a large public 3D VEM benchmark (BvEM) for cortical blood vessel segmentation across mouse, macaque, and human.
• Develop a zero-shot 3D segmentation method that elevates SAM to 3D VEM volumes without training or fine-tuning.
• Address 3D tracking challenges in VEM by leveraging multi-plane information and turning-point guidance.

提案手法

• Formulate 3D segmentation as a multi-seed tracking problem using SAM without annotated data.
• Introduce Tri-Plane Selection to choose the best tracking plane based on vessel flow direction and SAM confidence.
• Apply SAM-based tracking along the chosen plane with prompts updated from growing segmentation.
• Incorporate Recursive Redirection to identify turning points and enables long-term 3D tracking across slices.
• Use initial seeds from global thresholding and combine seeds across planes for final segmentation.

実験結果

リサーチクエスチョン

• RQ1 How can zero-shot segmentation models like SAM be extended to 3D VEM volumes without fine-tuning?
• RQ2 Does Tri-Plane selection improve long-term tracking stability for vascular structures in 3D volumes?
• RQ3 Can recursive redirection effectively capture vessel turning points to enhance 3D vessel continuity?
• RQ4 What is the performance of TriSAM on a large 3D VEM benchmark across species (mouse, macaque, human)?

主な発見

手法	学習データ	BvEM-マウス (Pre)	BvEM-マウス (Rec)	BvEM-マウス (Acc)	BvEM-マカク (Pre)	BvEM-マカク (Rec)	BvEM-マカク (Acc)	BvEM-ヒト (Pre)	BvEM-ヒト (Rec)	BvEM-ヒト (Acc)
TriSAM	External	84.12	66.75	59.28	78.41	74.97	62.14	31.35	25.57	16.39

• TriSAM achieves superior performance on the BvEM benchmark across mouse, macaque, and human.
• Tri-Plane selection significantly outperforms single-plane tracking, improving accuracy.
• Recursive Redirection yields the best overall accuracy among redirection strategies.
• MobileSAM provides faster inference and better accuracy than the original SAM in this 3D setting.