[Paper Review] Synaptic Cleft Segmentation in Non-Isotropic Volume Electron Microscopy of the Complete Drosophila Brain

Larissa Heinrich, Jan Funke+3 more|arXiv (Cornell University)|May 7, 2018

Advanced Electron Microscopy Techniques and Applications4 citations

TL;DR

This paper proposes a 3D-U-Net architecture trained via regression on a signed distance transform of manually annotated synaptic clefts in non-isotropic serial section transmission electron microscopy (ssTEM) data, achieving state-of-the-art performance on the CREMI challenge and enabling full-brain synaptic cleft prediction across 50 tera-voxels of the Drosophila brain with high consistency and speed.

ABSTRACT

Neural circuit reconstruction at single synapse resolution is increasingly recognized as crucially important to decipher the function of biological nervous systems. Volume electron microscopy in serial transmission or scanning mode has been demonstrated to provide the necessary resolution to segment or trace all neurites and to annotate all synaptic connections. Automatic annotation of synaptic connections has been done successfully in near isotropic electron microscopy of vertebrate model organisms. Results on non-isotropic data in insect models, however, are not yet on par with human annotation. We designed a new 3D-U-Net architecture to optimally represent isotropic fields of view in non-isotropic data. We used regression on a signed distance transform of manually annotated synaptic clefts of the CREMI challenge dataset to train this model and observed significant improvement over the state of the art. We developed open source software for optimized parallel prediction on very large volumetric datasets and applied our model to predict synaptic clefts in a 50 tera-voxels dataset of the complete Drosophila brain. Our model generalizes well to areas far away from where training data was available.

Motivation & Objective

• Address the challenge of accurate synaptic cleft segmentation in non-isotropic ssTEM data, which remains a bottleneck for large-scale insect connectomics.
• Overcome limitations of existing methods that underperform on non-isotropic data, especially for small, complex synapses in insect brains.
• Develop a deep learning framework that generalizes well across diverse brain regions despite sparse training data and non-isotropic resolution.
• Enable efficient, scalable prediction across tera-voxel-scale datasets using optimized parallel inference on distributed GPUs.
• Achieve performance comparable to human annotators on non-isotropic ssTEM data, a key milestone for automated connectome reconstruction.

Proposed method

• Design a 3D-U-Net architecture (DTU-2) optimized for non-isotropic ssTEM data, with reduced parameters and improved inductive bias for synaptic cleft detection.
• Train the network via regression on a signed Euclidean distance transform (SEDT) of binary synaptic cleft labels, scaled and passed through a tanh nonlinearity to stabilize training.
• Apply data augmentation techniques including elastic deformations, intensity variations, and ssTEM-specific artifacts (e.g., missing or noisy sections) to improve generalization.
• Use class-balanced sampling with 95% rejection of batches without synapses and rebalanced loss to penalize false negatives, addressing the extreme sparsity of synaptic clefts.
• Introduce an auxiliary loss for neuron boundary detection to improve localization, with both losses weighted equally during training.
• Deploy a distributed, GPU-accelerated inference pipeline using Dask and N5 format to process large volumes in parallel, with overlapping input blocks and non-overlapping output blocks.

Experimental results

Research questions

• RQ1 Can a 3D-U-Net architecture be effectively adapted to non-isotropic ssTEM data to improve synaptic cleft segmentation beyond current state-of-the-art methods?
• RQ2 Does training on a signed distance transform of manually annotated clefts lead to better generalization and robustness compared to standard pixel-wise classification?
• RQ3 Can the model generalize effectively to brain regions not present in the training data, such as the lamina or cortex, despite differences in morphology and imaging artifacts?
• RQ4 What is the scalability and efficiency of the inference pipeline when applied to full-brain datasets of 50 tera-voxels in size?
• RQ5 To what extent do the predicted synaptic cleft distributions match known biological patterns, such as those visualized by nc82 antibody labeling in confocal microscopy?

Key findings

• The proposed DTU-2 model outperforms previous architectures on the CREMI challenge, achieving a CREMI score that surpasses the state of the art despite using fewer parameters.
• The model generalizes well to unseen brain regions, producing accurate and consistent synaptic cleft predictions in areas like the lamina and cortex, even without training data from those regions.
• The prediction pipeline achieved an average throughput of ~3 mega-voxels per second per GPU, enabling full 50 tera-voxel Drosophila brain prediction in less than five days.
• The model's output, when convolved with a non-isotropic Gaussian PSF, reproduces the synaptic density patterns observed in confocal microscopy using the nc82 antibody, validating biological plausibility.
• The inclusion of an auxiliary loss for neuron boundary detection improved performance, particularly for the shallower DTU-1 architecture, indicating that multi-task learning enhances localization accuracy.
• The framework is open-source and publicly available under a permissive license, enabling reproducibility and broader adoption in connectomics research.