[Paper Review] Hindi Question Generation Using Dependency Structures

Kaveri Anuranjana, Vijjini Anvesh Rao+1 more|arXiv (Cornell University)|Jun 20, 2019

Topic Modeling22 references4 citations

TL;DR

This paper presents a rule-based Hindi question generation system that leverages karaka-dependency structures from a Hindi dependency parser and IndoWordNet for semantic role labeling to generate diverse, grammatically correct questions. By applying transformation rules based on karaka roles and applying syntactic and semantic filters, the system generates 112 questions from 30 sentences, significantly improving quality through pruning while maintaining high diversity.

ABSTRACT

Hindi question answering systems suffer from a lack of data. To address the same, this paper presents an approach towards automatic question generation. We present a rule-based system for question generation in Hindi by formalizing question transformation methods based on karaka-dependency theory. We use a Hindi dependency parser to mark the karaka roles and use IndoWordNet a Hindi ontology to detect the semantic category of the karaka role heads to generate the interrogatives. We analyze how one sentence can have multiple generations from the same karaka role's rule. The generations are manually annotated by multiple annotators on a semantic and syntactic scale for evaluation. Further, we constrain our generation with the help of various semantic and syntactic filters so as to improve the generation quality. Using these methods, we are able to generate diverse questions, significantly more than number of sentences fed to the system.

Motivation & Objective

• To address the scarcity of training data for Hindi question answering systems by automatically generating diverse, high-quality questions.
• To develop a rule-based question generation framework that does not require labeled training data, making it suitable for low-resource languages like Hindi.
• To improve question quality by applying syntactic and semantic filters to reduce overgeneration while preserving semantic and syntactic correctness.
• To evaluate the system using native Hindi speakers who rate generated questions on semantic and syntactic grounds.
• To explore the potential of dependency parsing and semantic ontologies for low-resource language NLP tasks in Indian languages.

Proposed method

• The system uses a Hindi dependency parser to identify karaka roles (e.g., k1 for agent, k2 for patient) in sentences, based on Paninian grammar and karaka theory.
• Interrogative words (e.g., kyon, kisne, kisne) are mapped to karaka roles based on case markers (direct vs. oblique case) to generate question templates.
• IndoWordNet is used to determine the semantic category of karaka role heads, enabling more contextually appropriate question formation.
• Rules are applied to transform each sentence into multiple question types (e.g., who, what, when, why) based on the karaka role being questioned.
• Syntactic filters are applied to remove questions violating word order (SOV), multiple interrogatives, or complex/compound sentence structures.
• Semantic filters are partially implemented due to tool limitations, but future work includes using IndoWordNet to resolve gender agreement and semantic consistency.

Experimental results

Research questions

• RQ1 Can a rule-based system generate diverse and grammatically correct Hindi questions without relying on large annotated datasets?
• RQ2 How effective are karaka-dependency roles and case markers in guiding question generation for Hindi?
• RQ3 To what extent do syntactic and semantic filters improve the quality of automatically generated questions?
• RQ4 How does overgeneration affect the quality of generated questions, and can filtering improve semantic and syntactic correctness?
• RQ5 Can dependency parsing and ontology-based semantic labeling enhance question generation in low-resource Indian languages?

Key findings

• The system generated 112 questions from just 30 input sentences, demonstrating a high degree of question diversity and overgeneration.
• Applying syntactic and semantic filters improved the quality of generated questions by 0.2 to 0.4 points on both semantic and syntactic evaluation scales.
• Native Hindi speakers rated the generated questions as semantically and syntactically valid, validating the effectiveness of the rule-based approach.
• The system’s reliance on dependency parsing means errors in parsing propagate to question generation, highlighting the need for robust parsers in long or complex sentences.
• Gender agreement and morphological agreement in verbs remain unresolved challenges, requiring future integration with IndoWordNet for dynamic verb form adjustment.
• The use of case markers (e.g., 'ne', 'ko') was critical in determining the correct interrogative word and verb agreement, especially in oblique case constructions.