[论文解读] Histo-Genomic Knowledge Distillation For Cancer Prognosis From Histopathology Whole Slide Images

Zhikang Wang, Yumeng Zhang+4 more|arXiv (Cornell University)|Mar 15, 2024

AI in cancer detection被引用 5

一句话总结

G-HANet 在训练过程中提炼 histo-genomic 知识，以提升基于 WSI 的癌症预后，从而在测试时实现无需基因组数据的单模态推断。

ABSTRACT

Histo-genomic multi-modal methods have recently emerged as a powerful paradigm, demonstrating significant potential for improving cancer prognosis. However, genome sequencing, unlike histopathology imaging, is still not widely accessible in underdeveloped regions, limiting the application of these multi-modal approaches in clinical settings. To address this, we propose a novel Genome-informed Hyper-Attention Network, termed G-HANet, which is capable of effectively distilling the histo-genomic knowledge during training to elevate uni-modal whole slide image (WSI)-based inference for the first time. Compared with traditional knowledge distillation methods (i.e., teacher-student architecture) in other tasks, our end-to-end model is superior in terms of training efficiency and learning cross-modal interactions. Specifically, the network comprises the cross-modal associating branch (CAB) and hyper-attention survival branch (HSB). Through the genomic data reconstruction from WSIs, CAB effectively distills the associations between functional genotypes and morphological phenotypes and offers insights into the gene expression profiles in the feature space. Subsequently, HSB leverages the distilled histo-genomic associations as well as the generated morphology-based weights to achieve the hyper-attention modeling of the patients from both histopathology and genomic perspectives to improve cancer prognosis. Extensive experiments are conducted on five TCGA benchmarking datasets and the results demonstrate that G-HANet significantly outperforms the state-of-the-art WSI-based methods and achieves competitive performance with genome-based and multi-modal methods. G-HANet is expected to be explored as a useful tool by the research community to address the current bottleneck of insufficient histo-genomic data pairing in the context of cancer prognosis and precision oncology.

研究动机与目标

• 通过在训练中利用 histo-genomic 对来降低对昂贵基因测序的依赖，从而提升模型。
• 开发一个单模态 WSI 预后模型，受益于多模态训练信号。
• 提出一个跨模态关联分支，用于从 WSIs 重建功能性基因组信息。
• 引入一个超注意力生存分支，将组织学信息与提炼的基因组信息融合用于预后。
• 在 TCGA 数据集上展示相较于最先进的单模态和多模态方法的改进。

提出的方法

• 将 WSIs 表示为按功能类别划分的补丁包和基因的集合。
• 使用带多头跨注意力的跨模态关联分支 (CAB) 来从 WSIs 重建功能性基因。
• 使用自归一化网络来建模高维度的基因功能，并从 Fp 重构 Xg。
• 引入一个 histo-genomic 超注意力 (HM) 模块，将形态学基础的注意力与基因组信息引导的注意力融合用于预后。
• 结合生存（负对数似然）和基因组重构（MSE 与缩放余弦误差）的联合损失进行训练。
• 在推理阶段，仅依赖 WSIs，因为部署时会舍弃基因组处理。

实验结果

研究问题

• RQ1 来自 histo-genomic 数据的多模态训练信号是否能够提升单模态 WSI 基于的预后？
• RQ2 如何将跨模态交互蒸馏到 WSI 表征中以提升生存预测？
• RQ3 形态学与提炼的基因组信号的超注意力融合是否优于现有的基于 WSI 的方法和多模态方法？
• RQ4 在用于基因信息特征构建的前 k 个补丁的选择对预后性能有何影响？

主要发现

方法	病理学	基因组	KD	BLCA	BRCA	GBMLGG	LUAD	UCEC	总体
Ours	✓	✓	✓	0.630 ± 0.032	0.664 ± 0.065	0.817 ± 0.022	0.612 ± 0.028	0.729 ± 0.050	0.690

• G-HANet 在五个 TCGA 数据集上与基因组基于和多模态方法相比具有竞争力的性能，总体 c-index 为 0.690。
• 在 GBMLGG 上，G-HANet 达到数据集特定的最佳 c-index 0.817。
• 在 BRCA 上，G-HANet 达到 0.664，超过了许多 WSI-based 基线。
• 与最先进的基于 WSI 的方法相比，G-HANet 显示出显著改进（例如 BRCA、LUAD、UCEC）。
• 该模型展示出强的跨数据集鲁棒性，并从将多模态知识提炼到单模态推理中受益。