[Paper Review] Framework for Question-Answering in Sanskrit through Automated Construction of Knowledge Graphs

Hrishikesh Terdalkar, Arnab Bhattacharya|arXiv (Cornell University)|Oct 11, 2023

Topic Modeling7 references4 citations

TL;DR

This paper presents a framework for building question-answering systems in Sanskrit by automatically constructing knowledge graphs from classical texts. It applies the system to three Sanskrit corpora—Mahābhārata, Rāmāyaṇa, and Bhāvaprakāśa Nighaṇṭu—achieving a 50% accuracy rate on factoid questions, while identifying key limitations in NLP pipelines and knowledge graph construction for Sanskrit.

ABSTRACT

Sanskrit (sa\d{m}sk\d{r}ta) enjoys one of the largest and most varied literature in the whole world. Extracting the knowledge from it, however, is a challenging task due to multiple reasons including complexity of the language and paucity of standard natural language processing tools. In this paper, we target the problem of building knowledge graphs for particular types of relationships from sa\d{m}sk\d{r}ta texts. We build a natural language question-answering system in sa\d{m}sk\d{r}ta that uses the knowledge graph to answer factoid questions. We design a framework for the overall system and implement two separate instances of the system on human relationships from mahābhārata and rāmāya\d{n}a, and one instance on synonymous relationships from bhāvaprakāśa nigha\d{n}\d{t}u, a technical text from āyurveda. We show that about 50% of the factoid questions can be answered correctly by the system. More importantly, we analyse the shortcomings of the system in detail for each step, and discuss the possible ways forward.

Motivation & Objective

• To address the challenge of extracting structured knowledge from Sanskrit literature due to linguistic complexity and lack of NLP tools.
• To design a scalable framework for automated knowledge graph construction tailored to specific semantic relationships in Sanskrit texts.
• To implement and evaluate a question-answering system that leverages these knowledge graphs to answer factoid questions in Sanskrit.
• To analyze the limitations of each component in the pipeline—especially in NLP processing and knowledge graph construction—for future improvements.

Proposed method

• The framework extracts specific types of relationships (e.g., human relationships in Mahābhārata, synonymous relationships in Bhāvaprakāśa Nighaṇṭu) from Sanskrit texts using rule-based and NLP techniques.
• It employs a pipeline combining morphological analysis, part-of-speech tagging, dependency parsing, and relation extraction to build structured knowledge graphs.
• The system uses a knowledge graph-based retrieval mechanism to answer natural language questions in Sanskrit by matching query patterns to graph-structured facts.
• The framework supports multiple text types: epics (Mahābhārata, Rāmāyaṇa) and technical treatises (Bhāvaprakāśa Nighaṇṭu), demonstrating adaptability.
• It applies a question classification and slot-filling approach to map questions to knowledge graph queries.
• Evaluation is conducted on factoid questions derived from the knowledge graphs, with answers validated against gold-standard annotations.

Experimental results

Research questions

• RQ1 Can automated construction of knowledge graphs from Sanskrit texts enable effective question-answering for factoid queries?
• RQ2 How accurate is a knowledge graph-based QA system when applied to diverse Sanskrit corpora, including epics and technical texts?
• RQ3 What are the primary bottlenecks in the NLP pipeline that limit performance in Sanskrit knowledge graph construction?
• RQ4 To what extent can rule-based and linguistic processing techniques compensate for the lack of pre-trained NLP models in Sanskrit?
• RQ5 How do different text genres (e.g., narrative vs. technical) affect the quality and usability of extracted knowledge graphs?

Key findings

• The system achieved a 50% accuracy rate in answering factoid questions across the three tested corpora, indicating moderate but promising performance.
• The highest performance was observed in the Bhāvaprakāśa Nighaṇṭu dataset, suggesting that technical and structured texts yield more reliable knowledge graphs.
• Morphological and syntactic complexity in Sanskrit significantly impacted the accuracy of entity and relation extraction, especially in narrative texts.
• The lack of standardized NLP tools for Sanskrit introduced errors in parsing and entity linking, which were major contributors to system failure.
• Dependency parsing and relation extraction were particularly sensitive to noise in the pre-processing pipeline, highlighting the need for robust linguistic tools.
• The study identifies key failure points in each stage of the pipeline, offering actionable insights for future improvements in Sanskrit NLP systems.