[Paper Review] A Closer Look at Domain Shift for Deep Learning in Histopathology

Karin Stacke, Gabriel Eilertsen+2 more|arXiv (Cornell University)|Sep 25, 2019

AI in cancer detection23 references47 citations

TL;DR

The paper studies domain shift in histopathology with CNN tumor classifiers across scanners, proposing a representation shift metric and evaluating augmentation/normalization techniques on CAMELYON17 data.

ABSTRACT

Domain shift is a significant problem in histopathology. There can be large differences in data characteristics of whole-slide images between medical centers and scanners, making generalization of deep learning to unseen data difficult. To gain a better understanding of the problem, we present a study on convolutional neural networks trained for tumor classification of H&E stained whole-slide images. We analyze how augmentation and normalization strategies affect performance and learned representations, and what features a trained model respond to. Most centrally, we present a novel measure for evaluating the distance between domains in the context of the learned representation of a particular model. This measure can reveal how sensitive a model is to domain variations, and can be used to detect new data that a model will have problems generalizing to. The results show how learning is heavily influenced by the preparation of training data, and that the latent representation used to do classification is sensitive to changes in data distribution, especially when training without augmentation or normalization.

Motivation & Objective

• Understand how domain shift affects CNN-based tumor classification in H&E whole-slide images across scanners.
• Assess how augmentation and normalization strategies influence performance and learned representations.
• Introduce a representation shift metric to quantify domain-induced changes in feature representations.

Proposed method

• Use CAMELYON17 H&E WSI patches from three scanners to simulate cross-domain shifts.
• Evaluate two small CNN architectures (Simple CNN and Mini-GoogLeNet) with and without augmentation/normalization.
• Apply color augmentation, stain normalization, and CycleGAN-based domain translation to test data.
• Visualize learned filters to interpret features the models rely on.
• Propose and compute a representation shift metric based on Wasserstein distance between per-filter activations in the last conv layer.

Experimental results

Research questions

• RQ1 How does cross-dataset generalization perform when training on one scanner and testing on unseen scanners?
• RQ2 What features do CNNs learn under different data processing strategies, and how does this affect robustness to domain shift?
• RQ3 Can a representation shift metric predict drops in cross-domain classification accuracy?

Key findings

• Without augmentation, cross-scanner generalization drops substantially (mean loss ~21.7 percentage points).
• Color augmentation improves cross-scanner generalization to a smaller drop (~4.75 p.p.).
• Stain normalization helps some unseen cases but can incur larger drops for certain scanners (~9.2 p.p.).
• CycleGAN yields best within-scanner performance but fails to generalize as well across scanners (up to ~11.45 p.p. drop).
• Feature visualizations show learned representations vary with data processing, with color-augmentation causing models to ignore color more completely.
• A negative correlation exists between representation shift and patch-level accuracy: larger shifts tend to accompany lower accuracy across domains.