[Paper Review] OmniScreen model predictions and evaluation of the MSK and TCGA datasets

Yi Kan Wang, L. Trlifaj Tydlitatova+2 more|arXiv (Cornell University)|Aug 18, 2024

Genetics, Bioinformatics, and Biomedical ResearchBiochemistry, Genetics and Molecular Biology3 citations

TL;DR

This study introduces OmniScreen, a unified AI model based on the Virchow2 foundation model that predicts 505 molecular biomarkers from H&E whole slide images across 15 common cancers. It achieves a mean AUROC of 0.89 for 80 high-performing biomarkers and enables high-throughput, cost-effective pan-cancer screening for genomic and phenotypic biomarkers without tissue destruction.

ABSTRACT

Molecular assays are standard of care for detecting genomic alterations in cancer prognosis and therapy selection but are costly, tissue-destructive and time-consuming. Artificial intelligence (AI) applied to routine hematoxylin and eosin (H&E)-stained whole slide images (WSIs) offers a fast and economical alternative for screening molecular biomarkers. We introduce OmniScreen, a high-throughput AI-based system leveraging Virchow2 embeddings extracted from 60,529 cancer patients with paired 489-gene MSK-IMPACT targeted biomarker panel and WSIs. Unlike conventional approaches that train separate models for each biomarker, OmniScreen employs a unified model to predict a broad range of clinically relevant biomarkers across cancers, including low-prevalence targets impractical to model individually. OmniScreen reliably identifies therapeutic targets and shared phenotypic features across common and rare tumors. We investigate the biomarker prediction probabilities and accuracies of OmniScreen in relation to tumor area, cohort size, histologic subtype alignment, and pathway-level morphological patterns. These findings underscore the potential of OmniScreen for routine clinical screening.

Motivation & Objective

• To develop a high-throughput, cost-effective method for screening multiple molecular biomarkers from routine H&E whole slide images (WSIs), avoiding tissue-destructive and time-consuming NGS assays.
• To overcome limitations of single-biomarker models by training a unified model capable of simultaneously predicting diverse biomarkers across multiple cancer types.
• To identify digital biomarkers in H&E WSIs that correlate with clinically relevant genomic alterations, including therapy targets, DNA repair defects, and genomic instability measures.
• To evaluate model performance on independent MSK and TCGA cohorts, demonstrating robustness and generalization across diverse cancer histologies and sample types.
• To enable broader clinical and pharmaceutical applications, including improved patient stratification, trial screening, and novel target discovery.

Proposed method

• The model uses Virchow2, a foundation model pre-trained on 3 million H&E WSIs, to extract 2,560-dimensional embeddings from 224×224 tissue tiles across whole slide images.
• Tissue tiles are filtered and selected based on morphological relevance before embedding generation, ensuring only relevant histological regions are used for representation.
• A unified multi-label classification head is trained to predict 505 gene-level biomarkers from the MSK-IMPACT panel, along with pathway activities and genomic instability metrics.
• The model is trained on 47,960 H&E WSIs from 38,984 patients, with validation performed on a tuned set and final evaluation on independent MSK and TCGA test sets.
• Model performance is evaluated using AUROC, sensitivity, specificity, and positive sample ratio thresholds to identify high-performing biomarkers.
• The approach enables simultaneous prediction of multiple biomarkers—such as TMB, MSI, CIN, and signaling pathway activity—without retraining for each target.

Experimental results

Research questions

• RQ1 Can a single, unified deep learning model accurately predict a broad panel of 505 clinically relevant molecular biomarkers from routine H&E WSIs across multiple cancer types?
• RQ2 How does the performance of the OmniScreen model compare to traditional NGS-based assays in predicting known genomic alterations, particularly in terms of AUROC and sensitivity?
• RQ3 To what extent can digital biomarkers derived from H&E images predict clinically actionable targets, DNA repair defects, and genomic instability phenotypes such as TMB and MSI?
• RQ4 Can the model generalize across primary and metastatic tumor samples, and how does performance vary between these sample types?
• RQ5 Can the model identify novel associations between histopathological features and specific cancer subtypes or therapeutic response markers?

Key findings

• The model identified 80 high-performing biomarkers with a mean AUROC of 0.89 across 15 most common cancer types, demonstrating strong predictive performance.
• Forty biomarkers showed strong associations with specific histologic subtypes, indicating morphological correlates of molecular heterogeneity.
• Fifty-eight biomarkers were linked to clinically relevant targets used in therapy selection and response prediction, enhancing clinical utility.
• The model successfully predicted activity of five canonical signaling pathways (TGF-β, RTK, HRD, mTOR, DDR), with AUC >0.75 for 14 out of 15 pathway and instability measures.
• Genomic instability metrics such as tumor mutation burden (TMB), microsatellite instability (MSI-H), and chromosomal instability (CIN) were predicted with AUC >0.75, supporting their use in immunotherapy and targeted therapy selection.
• The model achieved high sensitivity (>0.8) and specificity (>0.3) for 80 biomarkers, with at least 50 positive samples and a positive ratio >2%, indicating clinical feasibility.