[Paper Review] Weakly Supervised AI for Efficient Analysis of 3D Pathology Samples

Andrew H. Song, Mane Williams+9 more|arXiv (Cornell University)|Jul 27, 2023

AI in cancer detection4 citations

TL;DR

This paper introduces MAMBA, a weakly supervised deep learning framework for 3D pathology image analysis that enables patient-level risk stratification using volumetric tissue blocks from diverse imaging modalities. By leveraging multiple instance learning on 3D tissue blocks, MAMBA achieves an AUC of 0.86 for 5-year biochemical recurrence prediction in prostate cancer, outperforming 2D single-slice methods and reducing sampling bias through larger tissue volume analysis.

ABSTRACT

Human tissue and its constituent cells form a microenvironment that is fundamentally three-dimensional (3D). However, the standard-of-care in pathologic diagnosis involves selecting a few two-dimensional (2D) sections for microscopic evaluation, risking sampling bias and misdiagnosis. Diverse methods for capturing 3D tissue morphologies have been developed, but they have yet had little translation to clinical practice; manual and computational evaluations of such large 3D data have so far been impractical and/or unable to provide patient-level clinical insights. Here we present Modality-Agnostic Multiple instance learning for volumetric Block Analysis (MAMBA), a deep-learning-based platform for processing 3D tissue images from diverse imaging modalities and predicting patient outcomes. Archived prostate cancer specimens were imaged with open-top light-sheet microscopy or microcomputed tomography and the resulting 3D datasets were used to train risk-stratification networks based on 5-year biochemical recurrence outcomes via MAMBA. With the 3D block-based approach, MAMBA achieves an area under the receiver operating characteristic curve (AUC) of 0.86 and 0.74, superior to 2D traditional single-slice-based prognostication (AUC of 0.79 and 0.57), suggesting superior prognostication with 3D morphological features. Further analyses reveal that the incorporation of greater tissue volume improves prognostic performance and mitigates risk prediction variability from sampling bias, suggesting the value of capturing larger extents of heterogeneous 3D morphology. With the rapid growth and adoption of 3D spatial biology and pathology techniques by researchers and clinicians, MAMBA provides a general and efficient framework for 3D weakly supervised learning for clinical decision support and can help to reveal novel 3D morphological biomarkers for prognosis and therapeutic response.

Motivation & Objective

• To address the clinical limitations of 2D histopathology by enabling 3D volumetric analysis of whole tissue blocks.
• To overcome the impracticality of manual 3D image evaluation by pathologists due to data size and complexity.
• To develop a modality-agnostic deep learning framework that enables weakly supervised learning on 3D pathology images without requiring pixel-level annotations.
• To improve prognostic accuracy by capturing spatially heterogeneous 3D morphological features across larger tissue volumes.
• To provide a generalizable computational platform for clinical decision support in 3D pathology using minimal annotation.

Proposed method

• MAMBA employs a multiple instance learning (MIL) framework where each 3D tissue block is treated as a bag of patches, and the model learns to predict patient outcomes from weakly labeled bags.
• The framework uses a 3D spatiotemporal convolutional neural network and Swin transformer for feature encoding from 3D image patches.
• A global attention mechanism aggregates patch-level features into a bag-level prediction, enabling end-to-end training with patient-level labels only.
• Grad-CAM-based class activation maps (Grad-CAM++) are used to generate saliency heatmaps, visualizing which 3D regions contribute most to predictions.
• The method is trained on open-top light-sheet microscopy and micro-CT images of archived prostate cancer specimens, with risk stratification based on 5-year biochemical recurrence.
• Model interpretability is enhanced via integrated gradients (IG) to highlight discriminative morphological features in 3D volumes.

Experimental results

Research questions

• RQ1 Can a weakly supervised deep learning framework achieve superior prognostic performance in 3D pathology compared to traditional 2D single-slice analysis?
• RQ2 Does incorporating larger tissue volumes in 3D analysis reduce sampling bias and improve prediction consistency?
• RQ3 Can a modality-agnostic framework effectively process 3D pathology images from diverse imaging modalities such as light-sheet microscopy and microCT?
• RQ4 What 3D morphological features are most predictive of patient outcomes, and can they be interpreted through explainable AI techniques?
• RQ5 To what extent does volumetric analysis improve the area under the ROC curve (AUC) for 5-year biochemical recurrence prediction in prostate cancer?

Key findings

• MAMBA achieved an AUC of 0.86 for 5-year biochemical recurrence prediction in prostate cancer using light-sheet microscopy data, significantly outperforming 2D single-slice methods (AUC 0.79).
• For microCT data, MAMBA achieved an AUC of 0.74, compared to 0.57 for 2D single-slice analysis, demonstrating improved prognostic performance with 3D morphology.
• The incorporation of larger tissue volumes in 3D block-based analysis reduced risk prediction variability and mitigated sampling bias.
• Grad-CAM++ and integrated gradients (IG) visualizations revealed biologically plausible, spatially heterogeneous morphological features in 3D tissue blocks that contributed to patient outcome predictions.
• The model demonstrated generalization across imaging modalities, including light-sheet microscopy and microCT, confirming its modality-agnostic design.
• The interactive demo and open-source code enable community access and further development of 3D weakly supervised pathology tools.