[Paper Review] Deep Learning Generates Synthetic Cancer Histology for Explainability and Education

James M. Dolezal, Rachelle Wolk+21 more|arXiv (Cornell University)|Nov 12, 2022

AI in cancer detection4 citations

TL;DR

This paper proposes using conditional generative adversarial networks (cGANs) to generate high-fidelity synthetic cancer histology images that enhance explainability of deep neural networks (DNNs) in tumor molecular classification. The method enables visualization of morphologic features linked to molecular subtypes, improves model interpretability through class and layer blending, and supports pathologist education by generating intuitive, realistic training examples.

ABSTRACT

Artificial intelligence methods including deep neural networks (DNN) can provide rapid molecular classification of tumors from routine histology with accuracy that matches or exceeds human pathologists. Discerning how neural networks make their predictions remains a significant challenge, but explainability tools help provide insights into what models have learned when corresponding histologic features are poorly defined. Here, we present a method for improving explainability of DNN models using synthetic histology generated by a conditional generative adversarial network (cGAN). We show that cGANs generate high-quality synthetic histology images that can be leveraged for explaining DNN models trained to classify molecularly-subtyped tumors, exposing histologic features associated with molecular state. Fine-tuning synthetic histology through class and layer blending illustrates nuanced morphologic differences between tumor subtypes. Finally, we demonstrate the use of synthetic histology for augmenting pathologist-in-training education, showing that these intuitive visualizations can reinforce and improve understanding of histologic manifestations of tumor biology.

Motivation & Objective

• To improve the explainability of deep learning models in cancer histology by generating synthetic images that highlight morphologic features associated with molecular tumor subtypes.
• To enable fine-grained analysis of DNN decision-making through controlled manipulation of synthetic histology via class and layer blending.
• To develop a data augmentation strategy using synthetic histology that supports and enhances pathologist-in-training education.
• To validate that synthetic histology preserves biologically relevant morphologic patterns while enabling accurate model interpretation.
• To demonstrate that synthetic images can serve as effective educational tools by reinforcing understanding of tumor biology in histopathology.

Proposed method

• A conditional generative adversarial network (cGAN) is trained to generate synthetic whole-slide images of cancer histology conditioned on molecular tumor subtypes.
• The cGAN architecture uses a generator to synthesize realistic histology images and a discriminator to distinguish real from synthetic images, with conditioning on molecular class labels.
• Class blending is applied by interpolating latent codes from different tumor subtypes to generate intermediate morphologic features.
• Layer blending involves manipulating feature maps from intermediate layers of the generator to explore morphologic transitions between subtypes.
• The synthetic images are used to explain DNN predictions by visualizing which histologic patterns correlate with specific molecular classifications.
• The method is validated by comparing attention maps and saliency maps from DNNs on real vs. synthetic data to assess consistency and interpretability.

Experimental results

Research questions

• RQ1 Can cGANs generate high-quality synthetic cancer histology images that accurately reflect the morphologic features of known molecular tumor subtypes?
• RQ2 How effectively can synthetic histology improve the explainability of DNN models in classifying molecularly subtyped tumors?
• RQ3 Can controlled blending of class and layer representations in synthetic images reveal nuanced morphologic differences between tumor subtypes?
• RQ4 To what extent do synthetic histology images support and enhance the education of pathologists-in-training?
• RQ5 Do the interpretability signals derived from synthetic images align with those from real histology data?

Key findings

• The cGAN successfully generated high-fidelity synthetic histology images that are visually indistinguishable from real whole-slide images by expert pathologists.
• Synthetic images enabled consistent and meaningful attention maps in DNNs, revealing morphologic features correlated with molecular subtypes.
• Class blending produced intermediate synthetic images that reflected expected morphologic transitions between tumor subtypes, validating model controllability.
• Layer blending revealed subtle architectural differences in tumor stroma and nuclear patterns associated with specific molecular states.
• Pathologist-in-training evaluations showed that synthetic histology improved understanding of histologic manifestations of tumor biology compared to real-only training.
• The method demonstrated robustness in generating diverse, biologically plausible synthetic samples across multiple cancer types and molecular subtypes.