[论文解读] Unifying Neural Learning and Symbolic Reasoning for Spinal Medical Report Generation

Zhongyi Han, Benzheng Wei+2 more|arXiv (Cornell University)|Apr 28, 2020

Medical Imaging and Analysis参考文献 71被引用 5

一句话总结

本文提出了一种神经符号学习（NSL）框架，将深度神经网络的视觉感知与符号逻辑推理相结合，用于脊柱MRI报告生成。通过整合对抗性图网络进行语义分割，以及利用元解释学习实现无监督因果推理，NSL在脊柱结构检测和报告生成方面达到了最先进性能，在准确性和鲁棒性上优于现有方法。

ABSTRACT

Automated medical report generation in spine radiology, i.e., given spinal medical images and directly create radiologist-level diagnosis reports to support clinical decision making, is a novel yet fundamental study in the domain of artificial intelligence in healthcare. However, it is incredibly challenging because it is an extremely complicated task that involves visual perception and high-level reasoning processes. In this paper, we propose the neural-symbolic learning (NSL) framework that performs human-like learning by unifying deep neural learning and symbolic logical reasoning for the spinal medical report generation. Generally speaking, the NSL framework firstly employs deep neural learning to imitate human visual perception for detecting abnormalities of target spinal structures. Concretely, we design an adversarial graph network that interpolates a symbolic graph reasoning module into a generative adversarial network through embedding prior domain knowledge, achieving semantic segmentation of spinal structures with high complexity and variability. NSL secondly conducts human-like symbolic logical reasoning that realizes unsupervised causal effect analysis of detected entities of abnormalities through meta-interpretive learning. NSL finally fills these discoveries of target diseases into a unified template, successfully achieving a comprehensive medical report generation. When it employed in a real-world clinical dataset, a series of empirical studies demonstrate its capacity on spinal medical report generation as well as show that our algorithm remarkably exceeds existing methods in the detection of spinal structures. These indicate its potential as a clinical tool that contributes to computer-aided diagnosis.

研究动机与目标

• 解决从MRI图像中实现放射科医生水平的脊柱医学报告自动生成的挑战。
• 克服在空间关联模糊的情况下检测复杂且多变的脊柱结构的困难。
• 在标注数据有限的条件下，实现对检测到的异常的无监督因果效应分析。
• 通过结合视觉感知与逻辑推理，生成全面且临床可解释的报告。
• 开发一种可扩展、可解释且鲁棒的框架，以实现放射科临床部署。

提出的方法

• NSL框架采用生成对抗网络（GAN）与符号图推理模块相结合，以增强脊柱结构的语义分割。
• 对抗性图网络通过符号推理整合领域知识，提升在高度变异条件下对椎间盘、椎骨和神经孔的检测能力。
• 元解释学习通过建模检测到的异常之间的逻辑关系，实现无监督因果效应分析。
• 符号推理动态捕捉相邻脊柱结构之间的潜在空间相关性，提升分割的一致性。
• 统一模板根据符号推理结果填充疾病发现，生成完整且自然语言表达的诊断报告。
• 框架采用一阶逻辑编程实现稳健、可解释的知识表示与标注。

实验结果

研究问题

• RQ1 统一的神经符号框架是否能提升复杂MRI扫描中脊柱结构分割的准确性和鲁棒性？
• RQ2 在缺乏强监督的情况下，符号逻辑推理在多大程度上能有效建模脊柱异常之间的空间与因果关系？
• RQ3 将神经感知与符号推理相结合，能在多大程度上提升自动化医学报告生成的可解释性与临床实用性？
• RQ4 该框架是否能在具有不同解剖复杂度和病理特征的MRI扫描中实现良好泛化？
• RQ5 对抗性训练与符号推理的结合是否在分割与报告生成任务中优于纯粹的神经网络或符号基线模型？

主要发现

• NSL框架在脊柱结构分割中实现了96.1% ± 0.3的像素准确率和86.9% ± 2.3的Dice系数，优于消融模型。
• 与基础生成网络相比，符号图推理模块使像素准确率提升0.3%，Dice系数提升2.5%。
• 与基础自编码器相比，对抗性训练使像素准确率提升0.2%，Dice系数提升2.2%。
• 当与精确分割结合时，符号推理模型实现了100%的无监督标注准确率，展现出卓越的鲁棒性与可解释性。
• 完整NSL框架在分割与放射学分类任务中显著优于各消融版本，证明了所有组件的必要性。
• 该框架在具有高结构复杂性的挑战性MRI扫描中表现出强大的泛化能力，特殊病例的性能下降极小。