[论文解读] Fully transformer-based biomarker prediction from colorectal cancer histology: a large-scale multicentric study

Sophia J. Wagner, Daniel Reisenbüchler+29 more|arXiv (Cornell University)|Jan 23, 2023

Radiomics and Machine Learning in Medical Imaging被引用 5

一句话总结

本研究提出了一种完全基于Transformer的深度学习流程，用于从结直肠癌组织病理切片中端到端预测生物标志物，在手术切除标本和内镜活检样本上均实现了临床级性能。该模型在涵盖10个多中心队列的超过9,000名患者的数据上进行训练，对微卫星不稳定性（MSI）预测的敏感度达到97%，阴性预测值达到99%，相较于基于CNN的方法展现出更优的泛化能力和数据效率。

ABSTRACT

Background: Deep learning (DL) can extract predictive and prognostic biomarkers from routine pathology slides in colorectal cancer. For example, a DL test for the diagnosis of microsatellite instability (MSI) in CRC has been approved in 2022. Current approaches rely on convolutional neural networks (CNNs). Transformer networks are outperforming CNNs and are replacing them in many applications, but have not been used for biomarker prediction in cancer at a large scale. In addition, most DL approaches have been trained on small patient cohorts, which limits their clinical utility. Methods: In this study, we developed a new fully transformer-based pipeline for end-to-end biomarker prediction from pathology slides. We combine a pre-trained transformer encoder and a transformer network for patch aggregation, capable of yielding single and multi-target prediction at patient level. We train our pipeline on over 9,000 patients from 10 colorectal cancer cohorts. Results: A fully transformer-based approach massively improves the performance, generalizability, data efficiency, and interpretability as compared with current state-of-the-art algorithms. After training on a large multicenter cohort, we achieve a sensitivity of 0.97 with a negative predictive value of 0.99 for MSI prediction on surgical resection specimens. We demonstrate for the first time that resection specimen-only training reaches clinical-grade performance on endoscopic biopsy tissue, solving a long-standing diagnostic problem. Interpretation: A fully transformer-based end-to-end pipeline trained on thousands of pathology slides yields clinical-grade performance for biomarker prediction on surgical resections and biopsies. Our new methods are freely available under an open source license.

研究动机与目标

• 开发一种完全基于Transformer的深度学习流程，用于从结直肠癌组织病理切片中实现端到端生物标志物预测。
• 相较于现有的卷积神经网络（CNN）方法，提升性能、泛化能力、数据效率和可解释性。
• 评估仅在切除标本上进行训练是否能够实现在内镜活检组织上的高性能预测，以解决长期存在的诊断挑战。
• 在超过9,000名患者的大型多中心队列中对模型进行验证，涵盖10项独立研究。
• 将该方法开源发布，以支持临床转化和更广泛的应用。

提出的方法

• 该流程使用预训练的视觉Transformer编码器处理全切片图像的图像块。
• 一个专用的Transformer网络将局部图像块表征聚合为全局患者级嵌入，用于多目标预测。
• 模型采用弱监督学习方式端到端训练，其中切片级别的标签指导图像块级别的表征学习。
• 该架构支持从全切片图像中以患者为单位进行单目标和多目标生物标志物预测（例如，MSI、BRAF、KRAS）。
• 该方法在包含超过9,000名结直肠癌患者的大型多中心数据集上进行评估，涵盖10个独立队列。
• 通过标准指标（包括敏感度、阴性预测值和ROC曲线下面积）评估模型性能。

Figure 1: Workflow with pre-processing, model architecture and cohort overview. a) The data pre-processing pipeline with the steps whole slide image (WSI) digitization, tissue segmentation, tessellation into tiles, and stain normalization, b) the model architecture including the pre-trained feature

实验结果

研究问题

• RQ1 完全基于Transformer的架构是否能够在从结直肠癌组织病理切片中预测生物标志物方面超越最先进的CNN模型？
• RQ2 仅在手术切除标本上进行训练是否能够实现在内镜活检组织上的稳健预测？
• RQ3 大规模多中心训练队列在多大程度上提升了模型的泛化能力和数据效率？
• RQ4 基于Transformer的方法是否能够实现在结直肠癌MSI预测中的临床级性能？
• RQ5 该模型的可解释性与现有病理学深度学习方法相比如何？

主要发现

• 该完全基于Transformer的流程在手术切除标本上对微卫星不稳定性（MSI）预测的敏感度达到0.97，阴性预测值达到0.99。
• 尽管仅在切除标本上进行训练，该模型在内镜活检组织上仍表现出有效泛化，实现了临床级性能。
• 与最先进的基于CNN的模型相比，该方法在性能、泛化能力、数据效率和可解释性方面均表现更优。
• 该模型在多样化多中心数据集中保持了高性能，表明其对机构间差异具有强鲁棒性。
• 该方法在来自10个独立结直肠癌队列的超过9,000名患者中得到验证，显著增强了其临床相关性。
• 已完全训练的模型以开源许可证公开发布，以支持临床整合和进一步研究。

Figure 2: Evaluation of the prediction performance for the biomarkers MSI, BRAF, and KRAS in single-cohort and large-scale multi-centric experiments. Experimental results for MSI-H (a-c), BRAF (d,e), and KRAS (f,g) prediction: a) AUROC scores for single cohort experiments for all CRC cohorts ordered