[论文解读] A Survey of Multimodal Information Fusion for Smart Healthcare: Mapping the Journey from Data to Wisdom
本综述基于 DIKW 的多模态融合在智慧医疗中的应用，回顾技术、数据集、挑战，以及面向未来的 DIKW 对齐融合的通用框架。
Multimodal medical data fusion has emerged as a transformative approach in smart healthcare, enabling a comprehensive understanding of patient health and personalized treatment plans. In this paper, a journey from data to information to knowledge to wisdom (DIKW) is explored through multimodal fusion for smart healthcare. We present a comprehensive review of multimodal medical data fusion focused on the integration of various data modalities. The review explores different approaches such as feature selection, rule-based systems, machine learning, deep learning, and natural language processing, for fusing and analyzing multimodal data. This paper also highlights the challenges associated with multimodal fusion in healthcare. By synthesizing the reviewed frameworks and theories, it proposes a generic framework for multimodal medical data fusion that aligns with the DIKW model. Moreover, it discusses future directions related to the four pillars of healthcare: Predictive, Preventive, Personalized, and Participatory approaches. The components of the comprehensive survey presented in this paper form the foundation for more successful implementation of multimodal fusion in smart healthcare. Our findings can guide researchers and practitioners in leveraging the power of multimodal fusion with the state-of-the-art approaches to revolutionize healthcare and improve patient outcomes.
研究动机与目标
- 采用并调整 DIKW 模型应用于智慧医疗中的多模态融合。
- 从数据到智慧，综述数据模态与融合方法。
- 提供分类法与一个面向未来工作的 DIKW 对齐通用融合框架。
- 识别挑战并提出解决方案以指引研究与实践。
提出的方法
- 描述智慧医疗中的数据模态（电子病历 EHR、影像、可穿戴设备、基因组学、传感器、环境、行为）。
- 回顾跨特征选择、基于规则的系统、机器学习、深度学习和自然语言处理等的前沿融合技术。
- 提出 DIKW 一致的分类法和一个通用的融合框架。
- 综合与预测性、预防性、个性化和参与式（4P）医疗相关的挑战、趋势与未来方向。
实验结果
研究问题
- RQ1智慧医疗中的多模态融合使用了哪些模态，它们是如何表示的？
- RQ2用于融合多模态医疗数据的主要方法学途径有哪些？
- RQ3如何用 DIKW 为框架来组织现有技术并指导未来研究？
- RQ4在医疗保健中 DIKW 对齐的多模态融合会出现哪些挑战和未来方向？
主要发现
| 模态 | 数据类型 | 数据集 | 实例数量 | 属性数量 | 任务 | 受欢迎程度 |
|---|---|---|---|---|---|---|
| EHR | eICU Collaborative Research Database | 200,000 admissions | - | Varies | Various tasks, mainly diagnosis and prognosis | Medium |
| Imaging | MRNet | 1,370 exams | MRI data | - | Disease detection | Low |
| Imaging | RSNA Pneumonia Detection Challenge | 30,000 images | Pneumonia labels | - | Disease detection | Low |
| Imaging | MURA | 40,895 images | Abnormal/normal | - | Disease detection | Medium |
| Imaging | Pediatric Bone Age Challenge Dataset | Thousands of images | Bone age | - | Bone age estimation | Medium |
| Imaging | Indiana University Chest X-ray Collection | 8,000 images | Chest radiograph | DICOM images | Various tasks | Medium |
| Imaging | FastMRI | Thousands of scans | MRI data | - | Image reconstruction | Medium |
| Imaging | CheXpert | 224,316 images | 14 labels per image | - | Disease detection | High |
| Imaging | OASIS Brains Project | Varies with dataset | MRI and clinical data | - | Brain studies | High |
| Imaging | LIDC-IDRI | Over 1,000 patients | CT scans with marked-up lesions | - | Nodule detection | High |
| Imaging | TCIA | Millions of images | Various data types | - | Cancer research | High |
| Imaging | ChestX-ray8 | 108,948 images | 8 labels per image | - | Disease detection | High |
| Imaging | BraTS | Varies annually | MRI data | - | Tumor segmentation | High |
| Genomics, Imaging | TCGA | Thousands of patients | Genomic and clinical data | - | Cancer research | High |
| Genomics, Imaging, EHR | UK Biobank | 500,000 individuals | Various data types | - | Various tasks | Medium |
| Genomics, Imaging, EHR | ADNI | Thousands of patients | MRI and clinical data | - | Alzheimer’s research | High |
| Genomics, Imaging | ImageCLEFmed | Varies annually | Various data types | - | Various tasks | Low |
| Genomics, Imaging | Openi | 4.5 million images | Various data types | - | Various tasks | Low |
| Multimodality | PhysioNet | Various datasets | Various data types | - | Various tasks | High |
- 基于 DIKW 的表示（数据、信息、知识、智慧）提供了一个循环、具备反馈的智慧医疗多模态融合视角。
- 一个分类法将模态与融合技术（特征选择、基于规则、ML、深度学习、NLP）在 DIKW 框架内连接起来。
- 提出一个通用的 DIKW 对齐的多模态融合框架，以指导未来研究和实际部署。
- 在数据质量、隐私、安 全、临床整合、伦理以及结果解释方面存在 substantial 挑战。
- 大量数据集和模态（电子病历、影像、可穿戴、基因组、传感器、环境、行为）支持多模态融合研究。
- 未来方向强调医疗的 4P（预测、预防、个性化、参与式）。
更好的研究，从现在开始
从论文设计到论文写作，大幅缩短您的研究时间。
无需绑定信用卡
本解读由 AI 生成，并经人工编辑审核。