[论文解读] Integrative Analysis of Patient Health Records and Neuroimages via Memory-based Graph Convolutional Network

Xi Sheryl Zhang, Jingyuan Chou+1 more|arXiv (Cornell University)|Sep 17, 2018

Machine Learning in Healthcare参考文献 52被引用 6

一句话总结

本文提出MemGCN，一种基于记忆的图卷积神经网络，用于整合纵向电子健康记录（EHRs）和神经影像数据，实现帕金森病分类。通过利用记忆网络建模序列性EHR数据，并利用图卷积网络（GCN）从脑网络图中提取特征，MemGCN在PPMI数据集上实现了卓越的分类性能，证明了多模态数据融合在临床数据分析中的有效性。

ABSTRACT

With the arrival of the big data era, more and more data are becoming readily available in various real-world applications and those data are usually highly heterogeneous. Taking computational medicine as an example, we have both Electronic Health Records (EHR) and medical images for each patient. For complicated diseases such as Parkinson's and Alzheimer's, both EHR and neuroimaging information are very important for disease understanding because they contain complementary aspects of the disease. However, EHR and neuroimage are completely different. So far the existing research has been mainly focusing on one of them. In this paper, we proposed a framework, Memory-Based Graph Convolution Network (MemGCN), to perform integrative analysis with such multi-modal data. Specifically, GCN is used to extract useful information from the patients' neuroimages. The information contained in the patient EHRs before the acquisition of each brain image is captured by a memory network because of its sequential nature. The information contained in each brain image is combined with the information read out from the memory network to infer the disease state at the image acquisition timestamp. To further enhance the analytical power of MemGCN, we also designed a multi-hop strategy that allows multiple reading and updating on the memory can be performed at each iteration. We conduct experiments using the patient data from the Parkinson's Progression Markers Initiative (PPMI) with the task of classification of Parkinson's Disease (PD) cases versus controls. We demonstrate that superior classification performance can be achieved with our proposed framework, comparing with existing approaches involving a single type of data.

研究动机与目标

• 为解决整合异构、多模态临床数据（特别是EHRs与神经影像）以改善疾病理解的挑战。
• 利用记忆网络建模EHRs的序列性，捕捉神经影像采集前的纵向临床病史。
• 通过融合动态EHR表征与图卷积网络提取的空间脑网络特征，提升疾病分类性能。
• 通过支持多跳注意力机制，增强可解释性，揭示脑连接与临床事件中的疾病特异性模式。
• 证明多模态融合在分类帕金森病患者与对照组方面优于单模态方法。

提出的方法

• 使用图卷积网络（GCN）从基于感兴趣区（ROIs）的神经影像数据构建的患者特异性脑网络中提取特征。
• 采用记忆网络编码EHR中的序列临床病史，将随时间推移的患者事件存储为记忆向量。
• 在记忆网络上应用多跳读取与更新机制，迭代优化表征，提升上下文理解能力。
• 通过双线性匹配层或内积层，将记忆网络生成的上下文化EHR表征与GCN提取的神经影像特征进行融合。
• 采用度量学习结合成对距离优化，增强帕金森病患者与健康对照组之间的判别能力。
• 采用基于ChebNet的图卷积操作，实现在不规则脑网络拓扑结构上的高效且有效的特征学习。

Figure 1: Memory-based Graph Convolutional Network for brain connectivity graphs with clinical records. For simplicity, we depict the clinical records via a sequence of vectors in the figure. In practice, each clinical sequence is corresponding to a neuroimage acquisition.

实验结果

研究问题

• RQ1 统一的深度学习框架能否有效整合异构的EHR与神经影像数据，以提升帕金森病分类性能？
• RQ2 与单次读取机制相比，多跳记忆机制如何提升对EHR中纵向临床事件的表征能力？
• RQ3 通过模型的注意力机制揭示的帕金森病与健康对照组之间，在脑连接与临床事件模式方面有何显著差异？
• RQ4 通过MemGCN实现的多模态融合在多大程度上优于仅依赖EHR或神经影像的单模态模型？
• RQ5 模型内部的注意力与匹配层能否提供关于疾病进展模式的可解释性洞察？

主要发现

• 与仅使用EHR或仅使用神经影像的现有单模态方法相比，MemGCN在PPMI数据集上实现了更优的分类性能。
• 多跳记忆机制能够实现对序列性EHR数据更精细且上下文准确的表征学习，提升模型的判别能力。
• 模型的注意力图显示，帕金森病患者在记忆位置和特定ROIs上的注意力模式更窄、更集中，相较于对照组。
• 关键ROIs如右侧尾状核、壳核、丘脑和岛叶（Desikan-Killiany图谱中的第76–79位）显示出与疾病进展强相关且一致的激活模式。
• 双线性匹配与内积层生成了可解释的表征，凸显了PD患者与对照组在脑连接模式与临床事件时间上的群体差异。
• 即使在小样本队列数据集（如PPMI）上，该框架也表现出强劲性能，表明经进一步架构优化后，其在低数据场景下具有潜在应用价值。

Figure 2: Illustration of memory augmented graph convolution in a single hop (the $1$ -st hop). See Section II-C for the details.