[Paper Review] Nellie: Automated organelle segmentation, tracking, and hierarchical feature extraction in 2D/3D live-cell microscopy

Austin E.Y.T. Lefebvre, Gabriel Sturm+5 more|arXiv (Cornell University)|Mar 20, 2024

Cell Image Analysis Techniques5 citations

TL;DR

Nellie presents an automated, metadata-adaptive pipeline for segmentation, tracking, and multi-scale feature extraction of intracellular structures in 2D/3D live-cell microscopy with a user-friendly Napari GUI.

ABSTRACT

The analysis of dynamic organelles remains a formidable challenge, though key to understanding biological processes. We introduce Nellie, an automated and unbiased user-friendly pipeline for segmentation, tracking, and feature extraction of diverse intracellular structures. Nellie adapts to image metadata, eliminating user input. Nellie's preprocessing pipeline enhances structural contrast on multiple intracellular scales allowing for robust hierarchical segmentation of sub-organellar regions. Internal motion capture markers are generated and tracked via a radius-adaptive pattern matching scheme, and used as guides for sub-voxel flow interpolation. Nellie extracts a plethora of features at multiple hierarchical levels for deep and customizable analysis. Nellie features a point-and-click Napari-based GUI that allows for code-free operation and visualization, while its modular open-source codebase invites extension by experienced users. We demonstrate Nellie's wide variety of use cases with three examples: unmixing multiple organelles from a single channel using feature-based classification, training an unsupervised graph autoencoder on mitochondrial multi-mesh graphs to quantify latent space embedding changes following ionomycin treatment, and performing in-depth characterization and comparison of endoplasmic reticulum networks across different cell types and temporal frames.

Motivation & Objective

• Motivate automated, unbiased analysis of dynamic organelles to understand biological processes.
• Develop a metadata-adaptive pipeline that minimizes user input for segmentation, tracking, and feature extraction.
• Enable robust hierarchical segmentation that captures sub-organellar structures across scales.
• Provide a visual, code-free interface (Napari-based) and a modular open-source codebase for extensibility.

Proposed method

• Preprocessing that enhances structural contrast across intracellular scales to enable hierarchical segmentation.
• Radius-adaptive pattern matching to generate and track internal motion markers and guide sub-voxel flow interpolation.
• Hierarchical segmentation and feature extraction across multiple organizational levels of organelles.
• Feature extraction at multiple levels to support deep and customizable analyses.
• A Napari-based point-and-click GUI for code-free operation and visualization, plus an open-source modular codebase for extension.

Experimental results

Research questions

• RQ1 Can Nellie automatically segment and track diverse intracellular organelles without user input across 2D and 3D live-cell microscopy?
• RQ2 How effectively does metadata-adaptive preprocessing improve segmentation robustness across different imaging conditions?
• RQ3 Do multi-level (hierarchical) features enable improved downstream analyses such as classification, clustering, or network-based representations?
• RQ4 Can internal motion markers and sub-voxel flow interpolation improve tracking accuracy within crowded cellular contexts?
• RQ5 Are the provided use cases (unmixing organelles, graph autoencoder on mitochondrial graphs, ER network comparison) representative of Nellie’s versatility across cell types and temporal frames?

Key findings

• Nellie provides automated and unbiased segmentation, tracking, and hierarchical feature extraction without requiring user input for image metadata.
• The preprocessing and radius-adaptive tracking yield robust segmentation and motion capture across scales and structures.
• Demonstrations include feature-based unmixing of organelles from a single channel, an unsupervised graph autoencoder on mitochondrial graphs to quantify latent changes after ionomycin, and ER network comparisons across cell types and time.