[Paper Review] Domain-specific optimization and diverse evaluation of self-supervised models for histopathology

Jeremy Lai, Faruk Ahmed+14 more|arXiv (Cornell University)|Oct 20, 2023

AI in cancer detection4 citations

TL;DR

This paper proposes domain-specific self-supervised learning (SSL) methods for histopathology to improve foundation model performance across diverse tissue types and cancer diagnoses. Using a comprehensive benchmark of 17 tissue types and 12 cancer types across varying magnifications, the authors demonstrate that tailored SSL techniques significantly boost performance over standard methods, establishing high-quality foundation models for downstream tasks in pathology.

ABSTRACT

Task-specific deep learning models in histopathology offer promising opportunities for improving diagnosis, clinical research, and precision medicine. However, development of such models is often limited by availability of high-quality data. Foundation models in histopathology that learn general representations across a wide range of tissue types, diagnoses, and magnifications offer the potential to reduce the data, compute, and technical expertise necessary to develop task-specific deep learning models with the required level of model performance. In this work, we describe the development and evaluation of foundation models for histopathology via self-supervised learning (SSL). We first establish a diverse set of benchmark tasks involving 17 unique tissue types and 12 unique cancer types and spanning different optimal magnifications and task types. Next, we use this benchmark to explore and evaluate histopathology-specific SSL methods followed by further evaluation on held out patch-level and weakly supervised tasks. We found that standard SSL methods thoughtfully applied to histopathology images are performant across our benchmark tasks and that domain-specific methodological improvements can further increase performance. Our findings reinforce the value of using domain-specific SSL methods in pathology, and establish a set of high quality foundation models to enable further research across diverse applications.

Motivation & Objective

• To address data scarcity in histopathology by developing foundation models via self-supervised learning.
• To create a diverse, standardized benchmark spanning 17 tissue types, 12 cancer types, and multiple magnifications.
• To evaluate and optimize SSL methods specifically for histopathology, improving performance on patch-level and weakly supervised tasks.
• To establish a set of high-performing, generalizable foundation models for future research in pathology AI.
• To demonstrate that domain-specific methodological refinements significantly enhance SSL performance in histopathology.

Proposed method

• Developed a comprehensive benchmark with 17 unique tissue types and 12 unique cancer types across varying magnifications and task types.
• Applied and evaluated multiple self-supervised learning (SSL) methods on histopathology whole-slide images using contrastive and masked autoencoding approaches.
• Optimized SSL methods with domain-specific data augmentation, normalization, and training strategies tailored to histological image characteristics.
• Evaluated models on held-out patch-level classification and weakly supervised classification tasks to assess generalization.
• Used transfer learning to fine-tune foundation models on downstream tasks, measuring performance across diverse clinical scenarios.
• Conducted ablation studies to isolate the impact of domain-specific improvements on SSL performance.

Experimental results

Research questions

• RQ1 How do standard self-supervised learning methods perform across a diverse set of histopathology tasks spanning tissue types, cancers, and magnifications?
• RQ2 To what extent can domain-specific methodological adaptations improve SSL performance in histopathology?
• RQ3 Can foundation models pre-trained via domain-optimized SSL generalize effectively to downstream patch-level and weakly supervised classification tasks?
• RQ4 What components of data preprocessing and training strategy most significantly impact SSL performance in histopathology?
• RQ5 How does the performance of domain-optimized SSL models compare to standard SSL baselines across the benchmark?

Key findings

• Standard self-supervised learning methods achieve strong performance across the diverse histopathology benchmark when thoughtfully applied to histopathology images.
• Domain-specific methodological improvements—such as tailored data augmentation and normalization—further enhance model performance beyond standard SSL approaches.
• The proposed foundation models generalize effectively to held-out patch-level and weakly supervised classification tasks, demonstrating robust transferability.
• The benchmark of 17 tissue types and 12 cancer types across multiple magnifications provides a comprehensive and reproducible evaluation framework for future research.
• The study establishes a set of high-quality, performant foundation models for histopathology that can reduce data, compute, and expertise requirements for downstream model development.
• Performance gains from domain-specific optimization are quantitatively measurable and consistent across multiple task types and tissue categories.