[Paper Review] KGARevion: An AI Agent for Knowledge-Intensive Biomedical QA

Xiaorui Su, Yibo Wang+5 more|arXiv (Cornell University)|Oct 7, 2024

Quality and Safety in Healthcare4 citations

TL;DR

KGARevion is a knowledge graph (KG)-based LLM agent that enhances knowledge-intensive biomedical question answering by generating, verifying, and revising medical triplets against a grounded KG. It improves accuracy by over 5.2% on standard datasets and achieves a 10.4% improvement on new, complex medical QA benchmarks, demonstrating robustness and adaptability in multi-source medical reasoning.

ABSTRACT

Biomedical reasoning integrates structured, codified knowledge with tacit, experience-driven insights. Depending on the context, quantity, and nature of available evidence, researchers and clinicians use diverse strategies, including rule-based, prototype-based, and case-based reasoning. Effective medical AI models must handle this complexity while ensuring reliability and adaptability. We introduce KGARevion, a knowledge graph-based agent that answers knowledge-intensive questions. Upon receiving a query, KGARevion generates relevant triplets by leveraging the latent knowledge embedded in a large language model. It then verifies these triplets against a grounded knowledge graph, filtering out errors and retaining only accurate, contextually relevant information for the final answer. This multi-step process strengthens reasoning, adapts to different models of medical inference, and outperforms retrieval-augmented generation-based approaches that lack effective verification mechanisms. Evaluations on medical QA benchmarks show that KGARevion improves accuracy by over 5.2% over 15 models in handling complex medical queries. To further assess its effectiveness, we curated three new medical QA datasets with varying levels of semantic complexity, where KGARevion improved accuracy by 10.4%. The agent integrates with different LLMs and biomedical knowledge graphs for broad applicability across knowledge-intensive tasks. We evaluated KGARevion on AfriMed-QA, a newly introduced dataset focused on African healthcare, demonstrating its strong zero-shot generalization to underrepresented medical contexts.

Motivation & Objective

• Address the unique challenges of knowledge-intensive biomedical reasoning, which relies on diverse strategies like analogical, rule-based, and case-based reasoning.
• Overcome limitations of LLMs in biomedical QA, including hallucinations, poor retrieval, and lack of grounding in structured medical knowledge.
• Develop a system that integrates both non-codified LLM knowledge and structured, codified knowledge from medical KGs to improve reasoning accuracy and reliability.
• Ensure robustness against answer order and indexing biases common in multi-choice QA setups.
• Evaluate the model on both standard and newly curated, semantically complex medical QA benchmarks to assess generalization and performance under increasing difficulty.

Proposed method

• Use a four-step agent framework: Generate, Retrieve, Review, and Revise, to systematically process medical questions.
• Generate candidate medical triplets (subject-predicate-object) using a fine-tuned LLM prompted with the input question and KG embeddings.
• Retrieve relevant KG triplets using a vector search over pre-trained structural embeddings (e.g., from OGB-biokg or PrimeKG) to support retrieval.
• Review and verify generated triplets by cross-checking against a grounded medical KG to filter out hallucinated or incorrect facts.
• Revise the reasoning path by iteratively refining triplets based on feedback from the KG verification step.
• Leverage the KG not as a retrieval source but as a factual verifier, ensuring only medically accurate and relevant triplets contribute to the final answer.

Figure 1: a) Performance of LLMs on three new datasets introduced in this paper with questions of varying difficulty. b) Sample questions from new MedDDx-Basic, MedDDx-Intermediate, and MedDDx-Expert datasets.

Experimental results

Research questions

• RQ1 Can a KG-based agent improve the accuracy of LLMs in complex, knowledge-intensive biomedical question answering compared to standard RAG or LLM-only baselines?
• RQ2 How does the integration of structured KG verification enhance reasoning robustness in the presence of LLM hallucinations and retrieval errors?
• RQ3 To what extent does KGARevion maintain performance across varying levels of semantic complexity in medical QA, including expert-level questions?
• RQ4 How sensitive is KGARevion to answer ordering and labeling in multi-choice QA settings compared to standard LLMs?
• RQ5 Can the agent generalize across different KGs and LLM backbones while maintaining high accuracy and reliability?

Key findings

• KGARevion improves accuracy by over 5.2% on four gold-standard medical QA datasets compared to 15 baseline models, demonstrating strong performance on complex medical questions.
• On three newly curated, high-complexity medical QA datasets—MedDDx-Basic, MedDDx-Intermediate, and MedDDx-Expert—KGARevion achieves a 10.4% absolute accuracy improvement over baseline LLMs.
• KGARevion exhibits strong robustness to answer order and indexing in multi-choice settings, with only 2.59% and 3.86% accuracy loss for LLaMA3-8B and LLaMA3.1-8B, respectively, compared to 8.4% and 16.0% for pure LLMs.
• The agent’s performance remains stable and high even when using smaller KGs like OGB-biokg, outperforming models relying on larger KGs like PrimeKG, highlighting the effectiveness of its verification mechanism.
• The reasoning paths generated by KGARevion are explainable and grounded, with verified triplets forming a coherent, traceable chain of medical logic, such as linking chromosomal deletions to specific syndromes.
• KGARevion’s design enables systematic integration of both factual (structured) and tacit (non-codified) knowledge, allowing it to handle nuanced medical distinctions and semantically similar but clinically distinct concepts.

Figure 2: a) The overview of KGARevion . b ) The architecture of fine-tuning stage in the Review action, where embeddings get from KGs are structural embeddings, while concept embeddings from LLMs.