[Paper Review] Docs2KG: Unified Knowledge Graph Construction from Heterogeneous Documents Assisted by Large Language Models

Qiang Sun, Yuanyi Luo+6 more|UWA Profiles and Research Repository (University of Western Australia)|Jun 5, 2024

Topic Modeling4 citations

TL;DR

Docs2KG proposes a unified, LLM-augmented framework for constructing knowledge graphs from heterogeneous unstructured documents—such as PDFs, emails, web pages, and Excel files—by integrating multimodal data (text, tables, figures, images) through dynamic schema generation. It enables efficient, semantically rich querying and retrieval-augmented generation (RAG) by fusing structural and semantic relationships, significantly improving insight discovery and reducing hallucination in LLMs.

ABSTRACT

Even for a conservative estimate, 80% of enterprise data reside in unstructured files, stored in data lakes that accommodate heterogeneous formats. Classical search engines can no longer meet information seeking needs, especially when the task is to browse and explore for insight formulation. In other words, there are no obvious search keywords to use. Knowledge graphs, due to their natural visual appeals that reduce the human cognitive load, become the winning candidate for heterogeneous data integration and knowledge representation. In this paper, we introduce Docs2KG, a novel framework designed to extract multimodal information from diverse and heterogeneous unstructured documents, including emails, web pages, PDF files, and Excel files. Dynamically generates a unified knowledge graph that represents the extracted key information, Docs2KG enables efficient querying and exploration of document data lakes. Unlike existing approaches that focus on domain-specific data sources or pre-designed schemas, Docs2KG offers a flexible and extensible solution that can adapt to various document structures and content types. The proposed framework unifies data processing supporting a multitude of downstream tasks with improved domain interpretability. Docs2KG is publicly accessible at https://docs2kg.ai4wa.com, and a demonstration video is available at https://docs2kg.ai4wa.com/Video.

Motivation & Objective

• To address the challenge of extracting and integrating multimodal, heterogeneous unstructured data from enterprise data lakes, where 80% of data reside in unstructured formats.
• To overcome limitations of existing knowledge graph frameworks that rely on pre-designed schemas or target only specific document types or modalities.
• To enable dynamic, human-in-the-loop knowledge graph construction that maintains source references and supports both structural and semantic queries.
• To reduce hallucination in LLMs by grounding retrieval-augmented generation (RAG) in source-referenced, multimodal knowledge graphs.
• To provide a flexible, extensible, and publicly accessible framework for real-world deployment across domains like healthcare and research.

Proposed method

• Docs2KG employs a dual-path architecture combining deep learning-based document layout analysis and Markdown-structured parsing to maximize coverage across document types.
• It extracts entities and relationships from text, tables, figures, and images, then constructs a unified knowledge graph with intra-modal (e.g., 'has-child', 'before/after') and inter-modal (e.g., 'explain', 'same-time') relationships.
• The framework uses LLMs to generate semantic relationships between modalities, such as linking a table caption to its corresponding table via 'explain' or 'same-time'.
• A graph embedding model is applied to all nodes to enable semantic similarity search for RAG, retrieving top-k relevant nodes and expanding them via multi-hop queries.
• The system supports Cypher-based querying to retrieve document components related to specific events or time periods, such as 'events in 2011 and 2021'.
• The framework is designed to be dynamically updated and extended, allowing human-in-the-loop refinement and source traceability for RAG.

Experimental results

Research questions

• RQ1 How can a unified knowledge graph be constructed from heterogeneous unstructured documents across multiple formats, including PDFs, emails, web pages, and Excel files?
• RQ2 How can multimodal information—text, tables, figures, and images—be effectively integrated into a single, semantically coherent knowledge graph?
• RQ3 What role do dynamic, LLM-augmented semantic relationships play in improving knowledge graph interpretability and query expressiveness?
• RQ4 To what extent can the constructed knowledge graph enhance retrieval-augmented generation (RAG) by reducing hallucination and improving prompt grounding?
• RQ5 How does the integration of both structural and semantic relationships improve insight discovery in complex, multi-source document collections?

Key findings

• Docs2KG successfully integrated data from a PDF on Hong Kong’s population (2011–2021) and an Excel file (2021–2023), creating a unified knowledge graph that enabled cross-document insight extraction.
• The system demonstrated effective querying using Cypher to retrieve components related to events in 2011 and 2021, with results visualized as distinct nodes from each source (green for Excel, red for PDF).
• Semantic similarity search using node embeddings retrieved relevant tables and text chunks for the query 'population information from 2011 to 2021', including multi-hop expansion of related nodes.
• The framework achieved improved RAG performance by retrieving structurally and semantically proximate nodes, thereby grounding LLM responses in source-referenced knowledge.
• The system supports dynamic, extensible knowledge graph construction with human-in-the-loop capabilities, enabling iterative refinement and traceability to original documents.
• The public availability of Docs2KG at https://docs2kg.ai4wa.com enables reproducibility and real-world deployment across domains like healthcare and research.