[Paper Review] AI-driven 3D Spatial Transcriptomics

Cristina Almagro-Pérez, Andrew H. Song+14 more|ArXiv.org|Feb 25, 2025

Single-cell and spatial transcriptomics3 citations

TL;DR

VORTEX is an AI framework that predicts dense 3D spatial transcriptomics from 3D tissue morphology and minimal 2D ST data, enabling scalable, non-destructive volumetric gene expression maps.

ABSTRACT

A comprehensive three-dimensional (3D) map of tissue architecture and gene expression is crucial for illuminating the complexity and heterogeneity of tissues across diverse biomedical applications. However, most spatial transcriptomics (ST) approaches remain limited to two-dimensional (2D) sections of tissue. Although current 3D ST methods hold promise, they typically require extensive tissue sectioning, are complex, are not compatible with non-destructive 3D tissue imaging technologies, and often lack scalability. Here, we present VOlumetrically Resolved Transcriptomics EXpression (VORTEX), an AI framework that leverages 3D tissue morphology and minimal 2D ST to predict volumetric 3D ST. By pretraining on diverse 3D morphology-transcriptomic pairs from heterogeneous tissue samples and then fine-tuning on minimal 2D ST data from a specific volume of interest, VORTEX learns both generic tissue-related and sample-specific morphological correlates of gene expression. This approach enables dense, high-throughput, and fast 3D ST, scaling seamlessly to large tissue volumes far beyond the reach of existing 3D ST techniques. By offering a cost-effective and minimally destructive route to obtaining volumetric molecular insights, we anticipate that VORTEX will accelerate biomarker discovery and our understanding of morphomolecular associations and cell states in complex tissues. Interactive 3D ST volumes can be viewed at https://vortex-demo.github.io/

Motivation & Objective

• Motivate the need for true 3D spatial transcriptomics beyond 2D sections in heterogeneous tissues.
• Develop a scalable AI framework that learns morphomolecular links from diverse 3D morphology and 2D ST data.
• Enable fine-tuning on small VOIs to capture volume-specific morphomolecular correlations.
• Demonstrate 3D ST prediction across large tissue volumes using non-destructive 3D imaging modalities.
• Show cross-modal generalization to different imaging modalities and 2.5D (serial sections) contexts.

Proposed method

• Pretrain VORTEX on 3D morphology–2D ST pairs from diverse volumes and disease cohorts.
• Use cross-modal registration to align 3D tissue images, 2D tissue images, and 2D ST data.
• Employ four components: 2D/3D image encoders, transcriptomics encoder, and transcriptomics predictor.
• Train with a multi-task objective including contrastive loss (align embeddings) and ST reconstruction loss.
• Fine-tune on VOI-specific 2D ST data to incorporate volume-specific morphomolecular links.
• Optionally extend to 2.5D contexts using serial sections when 3D context is infeasible.

Experimental results

Research questions

• RQ1 Can VORTEX predict dense 3D ST from 3D morphology and limited 2D ST data?
• RQ2 Does incorporating 3D morphological context improve prediction over 2D-only approaches?
• RQ3 How does VOI-specific fine-tuning influence predictive accuracy and morphological coherence?
• RQ4 Is the model scalable to large tissue volumes and adaptable to different imaging modalities?
• RQ5 Can VORTEX generalize to 2.5D serial sections and unseen large volumes?

Key findings

• 3D+VOI training yields the best predictive performance across gene sets, with average PCC 0.46 (all genes), 0.57 (top 50), and 0.42 (marker genes).
• 3D+VOI consistently outperforms 3D and 2D training settings in PCC and SSIM metrics (e.g., SSIM all = 0.56 for 3D+VOI vs 0.51 for 3D and 0.50 for 2D).
• VORTEX captures expression variance and spatial autocorrelation more accurately with VOI fine-tuning, evidenced by higher Spearman’s ρ and improved Moran’s I/Geary’s C differences.
• 3D context enables predictions that align with morphological regions (tumor glands, stroma, benign glands) and reproduce known gene–morphology associations (e.g., EpCAM in tumor, ACTA2 in stroma).
• The framework scales to large tissue volumes (e.g., 6.62×8.85 mm2 with 1.71 mm depth) and generalizes across modalities (e.g., OTLS) and 2.5D serial-section data.
• VOI fine-tuning provides substantial gains even with minimal additional 2D ST data, enabling extrapolation beyond the original ST capture area.