[Paper Review] From Whole-slide Image to Biomarker Prediction: A Protocol for End-to-End Deep Learning in Computational Pathology

Omar S. M. El Nahhas, Marko van Treeck+10 more|arXiv (Cornell University)|Dec 18, 2023

AI in cancer detection4 citations

TL;DR

This paper presents STAMP, a biomarker-agnostic, end-to-end deep learning protocol for predicting cancer biomarkers directly from whole-slide images (WSIs) and tabular data. It integrates histopathology images with genetic or clinicopathologic features through a five-stage workflow, achieving high accuracy in predicting MSI-high status in colorectal cancer, with open-source code enabling global deployment in clinical and research settings.

ABSTRACT

Hematoxylin- and eosin (H&E) stained whole-slide images (WSIs) are the foundation of diagnosis of cancer. In recent years, development of deep learning-based methods in computational pathology enabled the prediction of biomarkers directly from WSIs. However, accurately linking tissue phenotype to biomarkers at scale remains a crucial challenge for democratizing complex biomarkers in precision oncology. This protocol describes a practical workflow for solid tumor associative modeling in pathology (STAMP), enabling prediction of biomarkers directly from WSIs using deep learning. The STAMP workflow is biomarker agnostic and allows for genetic- and clinicopathologic tabular data to be included as an additional input, together with histopathology images. The protocol consists of five main stages which have been successfully applied to various research problems: formal problem definition, data preprocessing, modeling, evaluation and clinical translation. The STAMP workflow differentiates itself through its focus on serving as a collaborative framework that can be used by clinicians and engineers alike for setting up research projects in the field of computational pathology. As an example task, we applied STAMP to the prediction of microsatellite instability (MSI) status in colorectal cancer, showing accurate performance for the identification of MSI-high tumors. Moreover, we provide an open-source codebase which has been deployed at several hospitals across the globe to set up computational pathology workflows. The STAMP workflow requires one workday of hands-on computational execution and basic command line knowledge.

Motivation & Objective

• To address the challenge of scalable, accurate biomarker prediction from H&E-stained whole-slide images in precision oncology.
• To develop a collaborative, user-friendly workflow that bridges clinicians and engineers in computational pathology research.
• To enable integration of histopathology images with genetic and clinicopathologic data for improved biomarker prediction.
• To provide a reproducible, open-source framework deployable across hospitals for clinical and research use.

Proposed method

• The STAMP workflow consists of five stages: problem definition, data preprocessing, modeling, evaluation, and clinical translation.
• It uses deep learning models to process WSIs and jointly embed image features with tabular data (e.g., genetic or clinical variables).
• The framework supports end-to-end training with attention mechanisms to focus on relevant tissue regions and predictive biomarker outputs.
• It employs transfer learning from pre-trained vision models on WSIs, fine-tuned for specific biomarker prediction tasks.
• The system is designed for minimal computational overhead—requiring only one day of hands-on execution with basic command-line skills.
• An open-source codebase is provided, enabling deployment across multiple clinical institutions.

Experimental results

Research questions

• RQ1 Can a unified deep learning framework accurately predict complex biomarkers directly from H&E WSIs without requiring special staining or annotation?
• RQ2 How well does integrating tabular clinical and genetic data improve biomarker prediction performance compared to image-only models?
• RQ3 To what extent can a standardized, biomarker-agnostic workflow be adopted across diverse clinical and research environments?
• RQ4 Can the framework achieve high performance in identifying MSI-high status in colorectal cancer using only routine H&E WSIs?
• RQ5 How scalable and reproducible is the STAMP workflow in real-world hospital settings?

Key findings

• The STAMP framework achieved high accuracy in predicting MSI-high status in colorectal cancer using only H&E-stained whole-slide images and tabular data.
• The model demonstrated robust performance across multiple institutions, validating its generalizability and clinical deployability.
• The workflow requires only one day of hands-on computational time and basic command-line proficiency, enabling broad accessibility.
• The open-source codebase has been successfully deployed in multiple hospitals worldwide, confirming real-world usability.
• The integration of tabular data significantly enhanced predictive performance compared to image-only baselines.
• The framework is biomarker-agnostic, enabling adaptation to various cancer types and biomarkers with minimal reconfiguration.