[论文解读] GAN-based Virtual Re-Staining: A Promising Solution for Whole Slide Image Analysis

Zhaoyang Xu, Huang, Xingru+3 more|arXiv (Cornell University)|Jan 13, 2019

Cell Image Analysis Techniques参考文献 20被引用 75

一句话总结

本文提出一种条件CycleGAN（cCGAN）来将H&E全.slide图像转换为类似IHC的图像以实现虚拟染色，结合子块级标签和真实感/SSIM损失以保持结构。

ABSTRACT

Histopathological cancer diagnosis is based on visual examination of stained tissue slides. Hematoxylin and eosin (H\&E) is a standard stain routinely employed worldwide. It is easy to acquire and cost effective, but cells and tissue components show low-contrast with varying tones of dark blue and pink, which makes difficult visual assessments, digital image analysis, and quantifications. These limitations can be overcome by IHC staining of target proteins of the tissue slide. IHC provides a selective, high-contrast imaging of cells and tissue components, but their use is largely limited by a significantly more complex laboratory processing and high cost. We proposed a conditional CycleGAN (cCGAN) network to transform the H\&E stained images into IHC stained images, facilitating virtual IHC staining on the same slide. This data-driven method requires only a limited amount of labelled data but will generate pixel level segmentation results. The proposed cCGAN model improves the original network \cite{zhu_unpaired_2017} by adding category conditions and introducing two structural loss functions, which realize a multi-subdomain translation and improve the translation accuracy as well. % need to give reasons here. Experiments demonstrate that the proposed model outperforms the original method in unpaired image translation with multi-subdomains. We also explore the potential of unpaired images to image translation method applied on other histology images related tasks with different staining techniques.

研究动机与目标

• 推动虚拟染色的需求，以克服H&E低对比度的局限性并减少对昂贵IHC的依赖。
• 开发一个多类别的无配对图像到图像翻译模型，以从H&E生成虚拟IHC。
• 通过专业化的损失函数确保翻译过程中的原始组织结构保真性。
• 通过虚拟染色提升组织分割和下游AI分析的效果。

提出的方法

• 将CycleGAN扩展为带有组织类别的条件输入（多子领域翻译）。
• 附加基于图块的分类器以强制正确的子领域表示。
• 引入真实感损失和SSIM损失，以在翻译过程中保持结构和纹理。
• 结合条件对抗损失、循环损失和分类损失来训练生成器、判别器和分类器。
• 优化一个加权损失L，以平衡cGAN、cycle、identity、class、clcyc、SSIM和真实感项（Eq. 11）。

实验结果

研究问题

• RQ1 无配对数据是否也能通过图像对图像翻译实现从H&E到IHC的准确多类别虚拟染色？
• RQ2 加入图块级分类信息是否能提升翻译质量和子域一致性？
• RQ3 真实感和SSIM损失是否在虚拟染色过程中保持组织结构？
• RQ4 所提出的损失对在获得正确染色风格的同时保持原始形态的影响？

主要发现

• 所提出损失的cCGAN相较于CycleGAN在虚拟染色质量上有所提升，尤其在训练数据有限的混合成分情形。
• 发现的最佳设置：gamma=0.5，alpha=0.5，beta=1。
• 量化结果显示不同类别的准确度存在差异，TF、HB、HF、TN由于数据平衡和复杂性而具有不同挑战。
• 病理学家评审表明相比CycleGAN，更好地保留内容且减少了虚假伪影。
• Zero-shot或无监督扩展被讨论为未来工作，以减少对标签的依赖。