[Paper Review] Virtual Histology with Photon Absorption Remote Sensing using a Cycle-Consistent Generative Adversarial Network with Weakly Registered Pairs

James E. D. Tweel, Benjamin R. Ecclestone+4 more|arXiv (Cornell University)|Jun 14, 2023

Cell Image Analysis Techniques4 citations

TL;DR

This paper proposes a label-free virtual histochemical staining method using photon absorption remote sensing (PARS) microscopy combined with a cycle-consistent generative adversarial network (CycleGAN) to generate H&E-like virtual histology images from unstained tissue sections. By applying a self-supervised Noise2Void denoising network and a novel mechanical scanning error correction algorithm to PARS data, the method achieves sub-micron structural resolution and produces virtual stains indistinguishable from gold-standard H&E staining in malignant skin and breast tissues.

ABSTRACT

Modern histopathology relies on the microscopic examination of thin tissue sections stained with histochemical techniques, typically using brightfield or fluorescence microscopy. However, the staining of samples can permanently alter their chemistry and structure, meaning an individual tissue section must be prepared for each desired staining contrast. This not only consumes valuable tissue samples but also introduces delays in essential diagnostic timelines. In this work, virtual histochemical staining is developed using label-free photon absorption remote sensing (PARS) microscopy. We present a method that generates virtually stained histology images that are indistinguishable from the gold standard hematoxylin and eosin (H&E) staining. First, PARS label-free ultraviolet absorption images are captured directly within unstained tissue specimens. The radiative and non-radiative absorption images are then preprocessed, and virtually stained through the presented pathway. The preprocessing pipeline features a self-supervised Noise2Void denoising convolutional neural network (CNN) as well as a novel algorithm for pixel-level mechanical scanning error correction. These developments significantly enhance the recovery of sub-micron tissue structures, such as nucleoli location and chromatin distribution. Finally, we used a cycle-consistent generative adversarial network CycleGAN architecture to virtually stain the preprocessed PARS data. Virtual staining is applied to thin unstained sections of malignant human skin and breast tissue samples. Clinically relevant details are revealed, with comparable contrast and quality to gold standard H&E-stained images. This work represents a crucial step to deploying label-free microscopy as an alternative to standard histopathology techniques.

Motivation & Objective

• To overcome the limitations of conventional histopathology, which requires permanent staining and consumes valuable tissue samples.
• To develop a non-destructive, label-free alternative to traditional histochemical staining using photon absorption remote sensing (PARS) microscopy.
• To generate high-fidelity virtual H&E-stained images from unlabeled, unstained tissue sections without chemical staining.
• To enable rapid, repeatable, and multi-contrast histological assessment from a single tissue section.

Proposed method

• Capturing ultraviolet absorption images via PARS microscopy from thin, unstained human tissue sections.
• Applying a self-supervised Noise2Void convolutional neural network (CNN) to denoise radiative and non-radiative PARS images.
• Implementing a novel pixel-level mechanical scanning error correction algorithm to enhance sub-micron structural fidelity.
• Using a cycle-consistent generative adversarial network (CycleGAN) to translate preprocessed PARS images into virtual H&E-stained appearances.
• Training the CycleGAN on weakly registered PARS and H&E image pairs to align tissue morphology across modalities.
• Validating the virtual staining output against gold-standard H&E-stained histology in malignant skin and breast tissue samples.

Experimental results

Research questions

• RQ1 Can PARS microscopy data be effectively preprocessed to recover sub-micron tissue structures such as nucleoli and chromatin distribution?
• RQ2 To what extent can a CycleGAN architecture generate virtual H&E-stained images that are perceptually indistinguishable from real H&E stains?
• RQ3 How does the combination of Noise2Void denoising and mechanical error correction improve the quality of label-free PARS images for virtual staining?
• RQ4 Can virtual staining via CycleGAN on weakly registered PARS and H&E pairs achieve clinically relevant contrast and resolution?
• RQ5 Does the proposed method preserve diagnostic-relevant histological features without chemical staining or tissue consumption?

Key findings

• The Noise2Void denoising network significantly improved signal-to-noise ratio in PARS images, enabling clearer visualization of sub-micron structures.
• The novel mechanical scanning error correction algorithm enhanced pixel-level alignment, improving structural accuracy in PARS-derived images.
• Virtual staining using CycleGAN produced images with contrast and texture quality comparable to gold-standard H&E staining.
• Clinically relevant histological features, including chromatin distribution and nucleolar morphology, were clearly resolved in the virtual stains.
• The method enabled high-fidelity virtual histology from a single unstained tissue section, eliminating the need for chemical staining and tissue consumption.
• The approach demonstrated feasibility for label-free, multi-contrast histopathology with potential for real-time diagnostic use.