[Paper Review] NucMM Dataset: 3D Neuronal Nuclei Instance Segmentation at Sub-Cubic Millimeter Scale

Zudi Lin, Donglai Wei+13 more|arXiv (Cornell University)|Jul 13, 2021

Cell Image Analysis TechniquesBiochemistry, Genetics and Molecular Biology37 references27 citations

TL;DR

This paper introduces the NucMM dataset, a large-scale 3D neuronal nuclei instance segmentation benchmark spanning sub-cubic millimeter volumes (0.1 mm³ zebrafish EM and 0.25 mm³ mouse uCT) with 170,000 and 7,000 nuclei respectively. It proposes a novel 3D hybrid-representation model, U3D-BCD, which jointly learns foreground mask, contour map, and signed distance transform to enable high-precision instance segmentation, achieving a 22% relative improvement in mean AP over state-of-the-art methods on the new benchmark.

ABSTRACT

Segmenting 3D cell nuclei from microscopy image volumes is critical for biological and clinical analysis, enabling the study of cellular expression patterns and cell lineages. However, current datasets for neuronal nuclei usually contain volumes smaller than $10^{ ext{-}3}\ mm^3$ with fewer than 500 instances per volume, unable to reveal the complexity in large brain regions and restrict the investigation of neuronal structures. In this paper, we have pushed the task forward to the sub-cubic millimeter scale and curated the NucMM dataset with two fully annotated volumes: one $0.1\ mm^3$ electron microscopy (EM) volume containing nearly the entire zebrafish brain with around 170,000 nuclei; and one $0.25\ mm^3$ micro-CT (uCT) volume containing part of a mouse visual cortex with about 7,000 nuclei. With two imaging modalities and significantly increased volume size and instance numbers, we discover a great diversity of neuronal nuclei in appearance and density, introducing new challenges to the field. We also perform a statistical analysis to illustrate those challenges quantitatively. To tackle the challenges, we propose a novel hybrid-representation learning model that combines the merits of foreground mask, contour map, and signed distance transform to produce high-quality 3D masks. The benchmark comparisons on the NucMM dataset show that our proposed method significantly outperforms state-of-the-art nuclei segmentation approaches. Code and data are available at https://connectomics-bazaar.github.io/proj/nucMM/index.html.

Motivation & Objective

• To address the lack of large-scale, multi-modality 3D neuronal nuclei datasets suitable for studying complex brain structures.
• To curate a benchmark dataset with significantly larger volume size and instance count than existing neuronal nuclei datasets.
• To develop a robust 3D instance segmentation model capable of handling high-density, low-contrast, and morphologically diverse nuclei across different imaging modalities.
• To quantitatively analyze the new challenges introduced by large-scale 3D neuronal nuclei segmentation in EM and uCT data.
• To establish a new state-of-the-art benchmark for 3D neuronal nuclei instance segmentation using the NucMM dataset.

Proposed method

• The NucMM dataset comprises two fully annotated 3D volumes: one 0.1 mm³ electron microscopy (EM) volume of a larval zebrafish brain with ~170,000 nuclei, and one 0.25 mm³ micro-CT (uCT) volume of a mouse visual cortex with ~7,000 nuclei.
• The proposed U3D-BCD model uses a 3D U-Net architecture to jointly predict three representations: foreground mask, instance contour map, and signed distance transform.
• At inference, a multi-target watershed segmentation algorithm combines the three predictions: seeds are identified via high foreground probability and distance value with low contour probability, and the signed distance map guides precise instance boundary detection.
• The model is trained using stochastic gradient descent (SGD) with cosine learning rate decay and data augmentations including brightness, flip, rotation, elastic transform, and missing parts.
• The signed distance transform is scaled using fixed parameters α=8 and β=50, and the model leverages consistency across representations to improve seed detection robustness.
• The decoding process is sensitive to thresholds on the foreground probability (τ4) and distance value (τ5), which significantly affect segmentation quality, indicating the importance of the signed distance map in capturing object structure.

Experimental results

Research questions

• RQ1 What are the key challenges in 3D neuronal nuclei instance segmentation at the sub-cubic millimeter scale, particularly in EM and uCT modalities?
• RQ2 How does the combination of multiple representation learning (foreground mask, contour map, signed distance transform) improve segmentation performance compared to single-representation models?
• RQ3 To what extent does the inclusion of the signed distance transform enhance the accuracy and robustness of instance boundary detection in high-density, low-contrast nuclear regions?
• RQ4 How does the proposed U3D-BCD model compare to state-of-the-art methods in terms of instance segmentation performance on large-scale, diverse 3D neuronal nuclei data?
• RQ5 What are the sensitivity and optimal configuration of decoding hyperparameters, particularly those related to the signed distance map and seed detection?

Key findings

• The NucMM dataset is over two magnitudes larger in volume size and instance count than previous neuronal nuclei datasets, with 0.1 mm³ and 0.25 mm³ volumes containing 170,000 and 7,000 nuclei respectively.
• The proposed U3D-BCD model achieves a mean average precision (mAP) of 0.894 on the NucMM-Z volume and 0.444 on the NucMM-M volume, outperforming all state-of-the-art methods by a relative margin of 22% on the overall benchmark.
• Cellpose [27] achieved only 0.569 mAP on NucMM-Z, while StarDist [34] scored 0.620, and U3D-BC [33] reached 0.670, demonstrating the superiority of the proposed U3D-BCD model.
• The U3D-BCD model ranks first in 6 out of 7 evaluation metrics, including 1st in mAP-50 and mAP-75 on both NucMM-Z and NucMM-M, indicating strong robustness across diverse challenges.
• Sensitivity analysis shows that the decoding threshold τ4 (foreground probability for valid regions) and τ5 (distance value threshold) have a significant impact on performance, with scores changing from 0.872 to 0.946 as τ4 varies from 0.1 to 0.4, highlighting the critical role of the signed distance map in accurate segmentation.
• The model is relatively insensitive to thresholds τ1 and τ2 (seed detection thresholds), with mAP-50 scores varying by less than 0.003 across a wide range of values, indicating robustness in seed localization.