[Paper Review] AI-powered virtual tissues from spatial proteomics for clinical diagnostics and biomedical discovery

Johann Wenckstern, Eeshaan Jain+10 more|arXiv (Cornell University)|Jan 10, 2025

Scientific Computing and Data Management7 citations

TL;DR

VirTues is a foundation-model framework for spatial proteomics that generalizes across cancer types and markers, enabling cross-study analysis, zero-shot predictions, and clinical decision support via tissue-based retrieval.

ABSTRACT

Spatial proteomics technologies have transformed our understanding of complex tissue architecture in cancer but present unique challenges for computational analysis. Each study uses a different marker panel and protocol, and most methods are tailored to single cohorts, which limits knowledge transfer and robust biomarker discovery. Here we present Virtual Tissues (VirTues), a general-purpose foundation model for spatial proteomics that learns marker-aware, multi-scale representations of proteins, cells, niches and tissues directly from multiplex imaging data. From a single pretrained backbone, VirTues supports marker reconstruction, cell typing and niche annotation, spatial biomarker discovery, and patient stratification, including zero-shot annotation across heterogeneous panels and datasets. In triple-negative breast cancer, VirTues-derived biomarkers predict anti-PD-L1 chemo-immunotherapy response and stratify disease-free survival in an independent cohort, outperforming state-of-the-art biomarkers derived from the same datasets and current clinical stratification schemes.

Motivation & Objective

• Address the heterogeneity and high dimensionality of multiplex spatial proteomics data across cancer and non-cancer tissues.
• Develop a transformer-based foundation model (VirTues) with marker and spatial attention and a novel tokenization scheme that preserves biological meaning.
• Enable zero-shot generalization to new datasets, markers, and cancer types without fine-tuning.
• Provide multi-scale tissue representations (cell, niche, tissue) for diagnostics, discovery, and retrieval-based clinical decision support.

Proposed method

• Introduce VirTues, a vision transformer with marker-specific and spatial attention that scales to hundreds of channels.
• Employ a novel tokenization scheme that combines protein language model embeddings with spatial channel patches and learnable cell summary tokens.
• Train a masked autoencoder (MAE) with channel-, marker-, and niche-level masking to learn cell-, niche-, and tissue-level representations.
• Generate cell, niche, and tissue summary tokens to enable downstream predictions across scales.
• Use sparse attention within transformer to manage high-dimensionality and maintain interpretability of marker interactions.
• Evaluate zero-shot generalization by testing on datasets/datasets with unseen markers; compare against CA-MAE and ResNet baselines.

Experimental results

Research questions

• RQ1 Can VirTues generalize across tissue types and unseen marker sets without task-specific fine-tuning?
• RQ2 How well can VirTues perform cell-type, niche-structure, and tissue-level predictions relative to established baselines?
• RQ3 Does the model provide interpretable marker- and spatial-attention insights that align with biological knowledge?
• RQ4 Can VirTues support clinical decision-making via retrieval of similar tissue cases with clinically relevant labels?

Key findings

• VirTues achieves strong performance across cellular, niche, and tissue-level tasks and outperforms CA-MAE and ResNet baselines on breast and lung cancer datasets.
• For breast cancer, VirTues attains ER status accuracy of 0.89 and cancer grade accuracy of 0.68, surpassing CA-MAE by 7.26% and 32.16% respectively (P-values reported for comparisons).
• For lung cancer, VirTues achieves cancer type accuracy of 0.87 and cancer grade accuracy of 0.63, with improvements over baselines of 11.72% and 16.21% respectively (P-values reported).
• VirTues demonstrates robust cell-type classification, including improved recall/F1 for rare cell populations (e.g., NK and B cells in breast cancer; vessel cells in lung cancer).
• Niche-level results show high accuracy for identifying structures such as suppressive expansions, TLS-like regions, and PDPN+ regions, surpassing baselines by notable margins (P-values provided).
• Tissue-level predictions include cancer type, grade, PAM50 subtyping for breast cancer, and cancer type/grade/relapse predictions for lung cancer, with multiple metrics surpassing baselines.