[Paper Review] Medical Knowledge Embedding Based on Recursive Neural Network for Multi-Disease Diagnosis

Jingchi Jiang, Huanzheng Wang+4 more|arXiv (Cornell University)|Sep 22, 2018

Machine Learning in Healthcare31 references3 citations

TL;DR

This paper proposes a Recursive Neural Knowledge Network (RNKN) that integrates first-order logic-based medical knowledge with recursive neural networks to generate diagnosis-oriented knowledge embeddings from Chinese Electronic Medical Records (CEMRs). Trained on annotated CEMRs, RNKN achieves superior diagnostic accuracy compared to classical machine learning models and Markov Logic Networks, with performance improving as evidence clarity increases.

ABSTRACT

The representation of knowledge based on first-order logic captures the richness of natural language and supports multiple probabilistic inference models. Although symbolic representation enables quantitative reasoning with statistical probability, it is difficult to utilize with machine learning models as they perform numerical operations. In contrast, knowledge embedding (i.e., high-dimensional and continuous vectors) is a feasible approach to complex reasoning that can not only retain the semantic information of knowledge but also establish the quantifiable relationship among them. In this paper, we propose recursive neural knowledge network (RNKN), which combines medical knowledge based on first-order logic with recursive neural network for multi-disease diagnosis. After RNKN is efficiently trained from manually annotated Chinese Electronic Medical Records (CEMRs), diagnosis-oriented knowledge embeddings and weight matrixes are learned. Experimental results verify that the diagnostic accuracy of RNKN is superior to that of some classical machine learning models and Markov logic network (MLN). The results also demonstrate that the more explicit the evidence extracted from CEMRs is, the better is the performance achieved. RNKN gradually exhibits the interpretation of knowledge embeddings as the number of training epochs increases.

Motivation & Objective

• To address the challenge of integrating symbolic medical knowledge with neural networks for improved multi-disease diagnosis.
• To develop a knowledge embedding framework that preserves semantic meaning and enables quantifiable reasoning in clinical decision support.
• To train a model on manually annotated Chinese Electronic Medical Records (CEMRs) to learn diagnosis-oriented knowledge representations.
• To evaluate the performance of the proposed model against classical machine learning models and Markov Logic Networks (MLN).
• To investigate how the interpretability of learned knowledge embeddings evolves with training epochs.

Proposed method

• The proposed RNKN model uses recursive neural networks to process medical knowledge expressed in first-order logic, enabling hierarchical representation learning.
• It learns high-dimensional, continuous vector representations (knowledge embeddings) that encode semantic and relational information from medical facts.
• The model is trained end-to-end on manually annotated Chinese Electronic Medical Records (CEMRs) to optimize diagnostic prediction.
• Knowledge embeddings and weight matrices are jointly learned during training, capturing both diagnostic relevance and logical structure.
• The recursive architecture allows the model to propagate information through logical substructures, modeling complex disease relationships.
• Interpretability of embeddings is assessed by analyzing their evolution across training epochs.

Experimental results

Research questions

• RQ1 Can a recursive neural network effectively learn diagnosis-oriented knowledge embeddings from first-order logic-based medical knowledge?
• RQ2 How does the diagnostic accuracy of RNKN compare to classical machine learning models and Markov Logic Networks (MLN)?
• RQ3 To what extent does the clarity of extracted evidence from CEMRs influence the model's performance?
• RQ4 How do the learned knowledge embeddings evolve in interpretability as training progresses?
• RQ5 Can the model capture and reason with complex, structured medical knowledge while maintaining numerical compatibility with deep learning?

Key findings

• RNKN achieves higher diagnostic accuracy than several classical machine learning models and Markov Logic Networks (MLN).
• The model's performance improves significantly with clearer and more explicit evidence extracted from Chinese Electronic Medical Records (CEMRs).
• As training progresses, the knowledge embeddings become increasingly interpretable, reflecting meaningful semantic and relational patterns.
• The joint learning of knowledge embeddings and weight matrices enables effective reasoning over structured medical knowledge.
• The recursive neural architecture successfully captures hierarchical and relational structures in medical knowledge for diagnosis.
• The results demonstrate the feasibility of combining symbolic logic with deep learning for clinical reasoning in multi-disease diagnosis.