[Paper Review] Cell-Type Prototype-Informed Neural Network for Gene Expression Estimation from Pathology Images

Kazuya Nishimura, Ryoma Bise+3 more|arXiv (Cornell University)|Mar 19, 2026

Single-cell and spatial transcriptomics0 citations

TL;DR

This work introduces a Cell-type Prototype-informed Neural Network (CPNN) that estimates slide- and patch-level gene expression from pathology images by leveraging cell-type prototypes derived from scRNA-seq data to regularize and interpret predictions.

ABSTRACT

Estimating slide- and patch-level gene expression profiles from pathology images enables rapid and low-cost molecular analysis with broad clinical impact. Despite strong results, existing approaches treat gene expression as a mere slide- or spot-level signal and do not incorporate the fact that the measured expression arises from the aggregation of underlying cell-level expression. To explicitly introduce this missing cell-resolved guidance, we propose a Cell-type Prototype-informed Neural Network (CPNN) that leverages publicly available single-cell RNA-sequencing datasets. Since single-cell measurements are noisy and not paired with histology images, we first estimate cell-type prototypes-mean expression profiles that reflect stable gene-gene co-variation patterns.CPNN then learns cell-type compositional weights directly from images and models the relationship between prototypes and observed bulk or spatial expression, providing a biologically grounded and structurally regularized prediction framework. We evaluate CPNN on three slide-level datasets and three patch-level spatial transcriptomics datasets. Across all settings, CPNN achieves the highest performance in terms of Spearman correlation. Moreover, by visualizing the inferred compositional weights, our framework provides interpretable insights into which cell types drive the predicted expression. Code is publicly available at https://github.com/naivete5656/CPNN.

Motivation & Objective

• Motivate estimating gene expression from whole-slide images as mixtures of cell-type expressions.
• Incorporate cell-type prototypes from scRNA-seq to regularize and guide estimation despite modality gaps.
• Develop a probabilistic model linking image-derived cell-type compositions to cell-level prototypes.
• Evaluate CP NN on slide-level and patch-level datasets to demonstrate performance and interpretability.

Proposed method

• Define slide-level expression as a weighted sum of cell-type prototypes with image-derived weights.
• Generate cell-type prototypes from scRNA-seq data via negative binomial regression with batch adjustment.
• Estimate patch-level compositional weights from image features using a learnable MLP and softmax to obtain cell-type proportions.
• Model observed gene expression with a Negative Binomial likelihood and apply modality-correction parameters alpha and beta.
• Incorporate a regularization term tying prototypes and deconvolution-derived weights to improve consistency and interpretability.
• Extend framework to patch-level prediction by replacing NB loss with a Pearson correlation–based loss for ST data.

Figure 1 : (a) Gene expression is observed together with histology image as the sum of unobserved cell-level expressions. Expression of individual cells is not directly measurable. (b) Cell-level gene expression profiles are obtained from a public single-cell RNA sequencing (scRNA-seq) database, whi

Experimental results

Research questions

• RQ1 Can cell-type prototypes derived from scRNA-seq provide biologically grounded priors to improve WSI-based gene expression estimation?
• RQ2 Does incorporating cell-type composition improve slide-level and patch-level expression accuracy across diverse datasets?
• RQ3 How do modality gaps between single-cell and bulk/spatial data affect performance, and can corrective terms mitigate them?
• RQ4 Do inferred cell-type weights offer interpretable insights into which cell types drive predictions?

Key findings

• CPNN achieves the highest Spearman correlation across three slide-level datasets (BRCA, KIRC, LUAD) and three patch-level datasets.
• Prototype-informed regularization and modality correction improve robustness against modality gaps.
• Visualized compositional weights align with known BRCA subtype biology, providing interpretable cell-type contributions.
• Integrating CP NN into patch-level models consistently improves SCC on CSCC, Her2st, and STNet datasets.
• Ablation studies show the importance of prototype initialization, modality correction, prototype updates, and regularization for performance.
• GRanularity studies indicate medium-to-fine cell-type labels perform best, with coarse labels reducing performance.

Figure 2 : Illustration of concept of Cell-type Prototype-informed Neural Network (CPNN). We estimate gene expression based on cell-type prototypes by modeling the relationship between observed gene expression and cell-level gene expression.