[Paper Review] Question Answering over Knowledge Graphs with Neural Machine Translation and Entity Linking

Daniel Diomedi, Aidan Hogan|arXiv (Cornell University)|Jul 6, 2021

Topic Modeling10 citations

TL;DR

This paper proposes ElNeuQA, a hybrid approach to Question Answering over Knowledge Graphs that combines Neural Machine Translation (NMT) for query template generation, Entity Linking (EL) for accurate entity identification, and Slot Filling (SF) for template instantiation. By decoupling entity resolution from query generation, the method significantly reduces entity-related errors compared to pure NMT models, achieving superior performance on Wikidata QA tasks despite continued reliance on seen query templates.

ABSTRACT

The goal of Question Answering over Knowledge Graphs (KGQA) is to find answers for natural language questions over a knowledge graph. Recent KGQA approaches adopt a neural machine translation (NMT) approach, where the natural language question is translated into a structured query language. However, NMT suffers from the out-of-vocabulary problem, where terms in a question may not have been seen during training, impeding their translation. This issue is particularly problematic for the millions of entities that large knowledge graphs describe. We rather propose a KGQA approach that delegates the processing of entities to entity linking (EL) systems. NMT is then used to create a query template with placeholders that are filled by entities identified in an EL phase. Slot filling is used to decide which entity fills which placeholder. Experiments for QA over Wikidata show that our approach outperforms pure NMT: while there remains a strong dependence on having seen similar query templates during training, errors relating to entities are greatly reduced.

Motivation & Objective

• To address the out-of-vocabulary problem in neural machine translation (NMT) for Knowledge Graph Question Answering (KGQA), particularly for rare or unseen entities.
• To reduce errors in KGQA caused by incorrect or missing entity translations in pure NMT-based systems.
• To improve robustness and accuracy in generating SPARQL queries from natural language questions by decoupling entity linking from query generation.
• To evaluate the effectiveness of combining NMT, entity linking, and slot filling in a modular pipeline for complex KGQA tasks.
• To demonstrate that entity-related errors are significantly reduced when entities are resolved via dedicated linking systems before query template filling.

Proposed method

• The system uses NMT to generate a query template with generic placeholders (e.g., <obj1>, <subj1>) instead of specific entities, reducing the risk of out-of-vocabulary issues.
• Entity Linking (EL) is performed via an ensemble of four systems (AIDA, DBpedia Spotlight, OpenTapioca, TagME), with voting and scoring to rank entity candidates for each mention.
• Sequence labelling is applied using a BiLSTM-CRF model with GloVe and Flair contextual embeddings to identify roles (e.g., subject, object) of phrases in the question.
• Slot Filling (SF) maps placeholders in the NMT-generated template to specific entities by matching sequence labels to EL results, using entity ranking for conflict resolution.
• A force-filling strategy is applied as a fallback: overlapping mentions, role-based matching, and sequential filling are used to maximize placeholder completion.
• The pipeline is trained on LC-QuAD 2.0 and a novel WikidataQA dataset, with gold-standard entity annotations for training and evaluation.

Experimental results

Research questions

• RQ1 Can decoupling entity resolution from query generation reduce entity-related errors in NMT-based KGQA systems?
• RQ2 How does the performance of a hybrid NMT-EL-SF approach compare to pure NMT models in generating accurate SPARQL queries for Wikidata?
• RQ3 To what extent does the use of entity linking mitigate the out-of-vocabulary problem in KGQA?
• RQ4 What is the impact of entity ranking and fallback strategies on the robustness of query generation?
• RQ5 Does the modular design of NMT, EL, and SF improve generalization on unseen or complex question templates?

Key findings

• The proposed ElNeuQA approach outperforms state-of-the-art pure NMT baselines on Wikidata QA, particularly in reducing entity-related errors.
• Despite strong dependence on seen query templates during training, the method drastically reduces errors due to unseen or misidentified entities.
• The use of entity linking significantly improves the accuracy of entity grounding, especially for rare or ambiguous entities not seen during NMT training.
• The ensemble EL system with voting and scoring mechanisms provides robust entity candidate ranking, enhancing downstream slot filling performance.
• The force-filling strategy effectively recovers from missing or misclassified entity mentions, improving overall query completeness.
• The results demonstrate that modular integration of NMT, EL, and SF leads to a more reliable and accurate KGQA pipeline than end-to-end NMT alone.