[Paper Review] Virtual histological staining of unlabeled autopsy tissue

Yuzhu Li, Nir Pillar+13 more|arXiv (Cornell University)|Aug 2, 2023

Molecular Biology Techniques and Applications4 citations

TL;DR

This study introduces a deep learning-based virtual histological staining method that transforms autofluorescence images of unlabeled, autolyzed autopsy tissue into high-fidelity H&E-like virtual stains, eliminating staining artifacts from delayed fixation. Trained on 0.7 TB of co-registered image data using a data-efficient registration-integrated neural network, the model successfully generates artifact-free, diagnostically reliable virtual H&E images from severely autolyzed tissues, including post-COVID-19 samples where traditional staining failed.

ABSTRACT

Histological examination is a crucial step in an autopsy; however, the traditional histochemical staining of post-mortem samples faces multiple challenges, including the inferior staining quality due to autolysis caused by delayed fixation of cadaver tissue, as well as the resource-intensive nature of chemical staining procedures covering large tissue areas, which demand substantial labor, cost, and time. These challenges can become more pronounced during global health crises when the availability of histopathology services is limited, resulting in further delays in tissue fixation and more severe staining artifacts. Here, we report the first demonstration of virtual staining of autopsy tissue and show that a trained neural network can rapidly transform autofluorescence images of label-free autopsy tissue sections into brightfield equivalent images that match hematoxylin and eosin (H&E) stained versions of the same samples, eliminating autolysis-induced severe staining artifacts inherent in traditional histochemical staining of autopsied tissue. Our virtual H&E model was trained using >0.7 TB of image data and a data-efficient collaboration scheme that integrates the virtual staining network with an image registration network. The trained model effectively accentuated nuclear, cytoplasmic and extracellular features in new autopsy tissue samples that experienced severe autolysis, such as COVID-19 samples never seen before, where the traditional histochemical staining failed to provide consistent staining quality. This virtual autopsy staining technique can also be extended to necrotic tissue, and can rapidly and cost-effectively generate artifact-free H&E stains despite severe autolysis and cell death, also reducing labor, cost and infrastructure requirements associated with the standard histochemical staining.

Motivation & Objective

• Address the critical challenge of poor staining quality in post-mortem tissue due to autolysis from delayed fixation.
• Overcome the resource-intensive, time-consuming, and labor-heavy nature of traditional histochemical staining in autopsy workflows.
• Develop a cost-effective, rapid, and scalable alternative to chemical H&E staining for autopsy tissue using artificial intelligence.
• Enable reliable histopathological diagnosis in global health crises where histopathology services are overwhelmed.
• Demonstrate the feasibility of virtual staining on severely autolyzed tissues, including those from COVID-19 fatalities.

Proposed method

• Employed a dual-stage image registration pipeline: first rigid registration at whole-slide image (WSI) level using maximum cross-correlation, followed by multi-modal affine registration at the patch level (3248×3248 px) to align autofluorescence and H&E images.
• Trained a deep neural network (DNN) on 16,159 co-registered image pairs (2048×2048 px after center-cropping) from autopsy tissues with minimal processing delays, using a data-efficient framework integrating registration and staining networks.
• Applied data augmentation via random flipping and rotation to 256×256 px patches during training, with normalization to zero mean and unit variance.
• Utilized a sCMOS camera (Leica DFC 9000 GTC) to capture autofluorescence images at 100 ms (DAPI) and 300 ms (TxRed) exposure times using specific filter cubes.
• Digitized histochemically stained H&E slides using a brightfield slide scanner (Leica Aperio AT2) for ground-truth pairing.
• Implemented a RegiStain framework with custom loss functions and training schedules to optimize virtual staining fidelity.

Experimental results

Research questions

• RQ1 Can a deep neural network accurately transform autofluorescence images of unlabeled autopsy tissue into virtual H&E stains that match the quality of traditional histochemical staining?
• RQ2 To what extent does the proposed virtual staining method reduce staining artifacts caused by autolysis in post-mortem tissues?
• RQ3 Does the virtual staining model maintain diagnostic consistency in nuclear and cytoplasmic features across severely autolyzed tissues, such as those from post-COVID-19 autopsies?
• RQ4 Can the virtual staining approach achieve comparable quantitative histological metrics (e.g., nuclear count, size) to traditional H&E staining?
• RQ5 How does the integration of image registration with the virtual staining network improve performance on challenging, non-ideal autopsy tissue samples?

Key findings

• The virtual H&E model achieved diagnostic-quality staining on severely autolyzed autopsy tissues, including post-COVID-19 samples, where traditional H&E staining failed due to poor dye binding and staining artifacts.
• Paired t-tests showed no statistically significant difference (p > 0.05) between virtual and histochemically stained images for both nuclear count per field of view and average nuclear size, indicating high quantitative consistency.
• Pathologist evaluations revealed no significant difference in staining quality metrics between virtual and traditional H&E stains, confirming diagnostic equivalence in visual assessment.
• The model successfully accentuated nuclear, cytoplasmic, and extracellular features in tissues with pronounced autolysis, such as vacuolation and cytoplasmic basophilia, which typically confound traditional staining.
• The training dataset comprised over 0.7 TB of image data, with 16,159 registered image pairs used to train the model, enabling robust generalization to unseen autopsy samples.
• The method significantly reduces labor, cost, and infrastructure demands by eliminating chemical reagents, staining procedures, and specialized lab facilities required in conventional histopathology workflows.