[Paper Review] Companion Animal Disease Diagnostics based on Literal-aware Medical Knowledge Graph Representation Learning

Van Thuy Hoang, Sang Thanh Nguyen+4 more|arXiv (Cornell University)|Aug 31, 2023

Biomedical Text Mining and OntologiesBiochemistry, Genetics and Molecular Biology3 citations

TL;DR

This paper proposes LiteralKG, a knowledge graph embedding model that fuses heterogeneous literal information (e.g., symptoms, text descriptions, treatments) with graph structure via gated fusion and attentive message passing to improve companion animal disease diagnosis. The model achieves state-of-the-art performance on link prediction, outperforming baselines by leveraging self-supervised pre-training on graph structure without labels.

ABSTRACT

Knowledge graph (KG) embedding has been used to benefit the diagnosis of animal diseases by analyzing electronic medical records (EMRs), such as notes and veterinary records. However, learning representations to capture entities and relations with literal information in KGs is challenging as the KGs show heterogeneous properties and various types of literal information. Meanwhile, the existing methods mostly aim to preserve graph structures surrounding target nodes without considering different types of literals, which could also carry significant information. In this paper, we propose a knowledge graph embedding model for the efficient diagnosis of animal diseases, which could learn various types of literal information and graph structure and fuse them into unified representations, namely LiteralKG. Specifically, we construct a knowledge graph that is built from EMRs along with literal information collected from various animal hospitals. We then fuse different types of entities and node feature information into unified vector representations through gate networks. Finally, we propose a self-supervised learning task to learn graph structure in pretext tasks and then towards various downstream tasks. Experimental results on link prediction tasks demonstrate that our model outperforms the baselines that consist of state-of-the-art models. The source code is available at https://github.com/NSLab-CUK/LiteralKG.

Motivation & Objective

• To address the limitation of existing knowledge graph embedding models that overlook diverse literal information in electronic medical records (EMRs) for animal disease diagnosis.
• To develop a unified representation learning framework that effectively fuses heterogeneous entity attributes (e.g., numerical, textual) with graph structure.
• To improve diagnostic performance by integrating self-supervised pre-training on graph structure to learn meaningful representations without labeled data.
• To investigate how different GNN aggregators and architectural components influence representation learning in heterogeneous medical KGs.

Proposed method

• Construct a medical knowledge graph (MKG) from EMRs collected across 31 companion animal hospitals, incorporating entities and relations from clinical notes, symptoms, treatments, and demographics.
• Apply a gate network to fuse diverse literal features—such as textual descriptions, numerical values, and categorical attributes—into unified entity representations.
• Employ an attentive message passing mechanism with residual connections to capture both local and global graph structures, enhancing feature propagation.
• Design a self-supervised pre-training task based on negative sampling of triplets to learn structural proximity in the KG without requiring labeled data.
• Fine-tune the pre-trained model on downstream link prediction tasks to evaluate diagnostic representation quality.
• Utilize multiple GNN aggregators (GCN, GraphSAGE, Bi-Interaction, GIN) to compare performance and analyze architectural impacts on representation learning.

Figure 1: An example of a sub-graph from our knowledge graph. The circles and the rectangles surrounding entities denote entities and their attributes, respectively. Several entities, such as $M$ , $A$ , $R$ , $O$ , and $D$ , denote the Medical records, Animals, Drugs, Treatment code, and Disease, r

Experimental results

Research questions

• RQ1 RQ1: Can fusing different types of literal information and graph structure improve representation learning for discovering complex relations in medical KGs?
• RQ2 RQ2: Which encoder architectures (e.g., GIN, Bi-Interaction) best capture heterogeneous features and structural complexity in medical KGs?
• RQ3 RQ3: Can a self-supervised pre-training task generate effective representations that enhance performance on downstream diagnostic tasks like link prediction?

Key findings

• LiteralKG achieved an F1 score of 0.8712 on link prediction using the Bi-Interaction aggregator, outperforming all baselines.
• The model improved F1 from 0.8021 to 0.8612 and accuracy from 0.8013 to 0.8490 when using pre-trained weights with the GIN aggregator.
• A 1-layer LiteralKG model achieved peak performance on the Bi-Interaction aggregator, suggesting that deeper stacks may lead to over-smoothing in this architecture.
• GIN aggregators showed superior scalability with depth, maintaining performance up to 8 layers due to their 1D-WL isomorphism power, which mitigates over-smoothing.
• Residual connections improved F1 from 0.8174 to 0.8378, indicating their effectiveness in preserving feature integrity during message passing.
• The pre-training phase significantly enhanced downstream performance, validating the utility of self-supervised learning for medical KG representation learning.

Figure 2: The overall architecture of LiteralKG. The entities and attributes are fused into unified vectors through a gate function $g(\cdot)$ . The unified vectors are then passed through attentive embedding propagation layers to generate the output representations. The output representations are t