[Paper Review] Unifying Large Language Models and Knowledge Graphs: A Roadmap

Shirui Pan, Linhao Luo+4 more|arXiv (Cornell University)|Jun 14, 2023

Topic Modeling257 references100 citations

TL;DR

The paper offers a three-framework roadmap to unify LLMs and knowledge graphs: KG-enhanced LLMs, LLM-augmented KGs, and synergized LLMs + KGs, with detailed categorization and future directions.

ABSTRACT

Large language models (LLMs), such as ChatGPT and GPT4, are making new waves in the field of natural language processing and artificial intelligence, due to their emergent ability and generalizability. However, LLMs are black-box models, which often fall short of capturing and accessing factual knowledge. In contrast, Knowledge Graphs (KGs), Wikipedia and Huapu for example, are structured knowledge models that explicitly store rich factual knowledge. KGs can enhance LLMs by providing external knowledge for inference and interpretability. Meanwhile, KGs are difficult to construct and evolving by nature, which challenges the existing methods in KGs to generate new facts and represent unseen knowledge. Therefore, it is complementary to unify LLMs and KGs together and simultaneously leverage their advantages. In this article, we present a forward-looking roadmap for the unification of LLMs and KGs. Our roadmap consists of three general frameworks, namely, 1) KG-enhanced LLMs, which incorporate KGs during the pre-training and inference phases of LLMs, or for the purpose of enhancing understanding of the knowledge learned by LLMs; 2) LLM-augmented KGs, that leverage LLMs for different KG tasks such as embedding, completion, construction, graph-to-text generation, and question answering; and 3) Synergized LLMs + KGs, in which LLMs and KGs play equal roles and work in a mutually beneficial way to enhance both LLMs and KGs for bidirectional reasoning driven by both data and knowledge. We review and summarize existing efforts within these three frameworks in our roadmap and pinpoint their future research directions.

Motivation & Objective

• Propose a forward-looking roadmap to unify LLMs and KGs and leverage their complementary strengths.
• Provide a fine-grained categorization and review of existing work within each integration framework.
• Summarize advances in LLMs and KGs, including multi-modal knowledge graphs and state-of-the-art models.
• Highlight challenges and outline promising directions for future research.

Proposed method

• Define three general frameworks for unifying LLMs and KGs: KG-enhanced LLMs, LLM-augmented KGs, and Synergized LLMs + KGs.
• Develop fine-grained categorizations within each framework (pre-training, inference, interpretability; embedding, completion, construction, KG-to-text, QA; knowledge representation and reasoning).
• Review existing methods and taxonomies across knowledge integration techniques, prompts, retrieval, and instruction-tuning.
• Synthesize challenges and future research directions to guide subsequent work in the field.

Experimental results

Research questions

• RQ1 How can knowledge graphs and large language models be integrated to overcome hallucination and interpretability issues?
• RQ2 What integration strategies best leverage KG structure during pre-training, inference, and instruction-tuning?
• RQ3 How can LLMs augment KG tasks such as embedding, completion, construction, and KG-to-text generation?
• RQ4 What are the key challenges and future directions for synergizing LLMs and KGs in bidirectional reasoning?

Key findings

• Provide a structured road map with three frameworks to unify LLMs and KGs.
• Offer a fine-grained taxonomy of research within each framework and summarize representative methods.
• Discuss state-of-the-art LLMs and evolving KGs, including multi-modal knowledge graphs.
• Identify challenges such as knowledge updating, interpretability, and zero-shot transfer, and propose directions for future work.