[Paper Review] Toward Subgraph-Guided Knowledge Graph Question Generation with Graph Neural Networks

Yu Chen, Lingfei Wu+1 more|arXiv (Cornell University)|Apr 13, 2020

Topic Modeling92 references21 citations

TL;DR

This paper proposes a bidirectional Graph2Seq model enhanced with a node-level copying mechanism to generate natural language questions from knowledge graph (KG) subgraphs, preserving structural and attribute information. The method outperforms prior approaches on two benchmarks, achieving state-of-the-art results and significantly boosting question answering via data augmentation.

ABSTRACT

Knowledge graph (KG) question generation (QG) aims to generate natural language questions from KGs and target answers. Previous works mostly focus on a simple setting which is to generate questions from a single KG triple. In this work, we focus on a more realistic setting where we aim to generate questions from a KG subgraph and target answers. In addition, most of previous works built on either RNN-based or Transformer based models to encode a linearized KG sugraph, which totally discards the explicit structure information of a KG subgraph. To address this issue, we propose to apply a bidirectional Graph2Seq model to encode the KG subgraph. Furthermore, we enhance our RNN decoder with node-level copying mechanism to allow directly copying node attributes from the KG subgraph to the output question. Both automatic and human evaluation results demonstrate that our model achieves new state-of-the-art scores, outperforming existing methods by a significant margin on two QG benchmarks. Experimental results also show that our QG model can consistently benefit the Question Answering (QA) task as a mean of data augmentation.

Motivation & Objective

• To address the limitation of existing knowledge graph question generation (KG-QG) methods that focus only on single triples or linearized subgraphs, which discard rich structural information.
• To improve question generation quality by leveraging the full structural and attribute information of KG subgraphs through a graph-based encoder.
• To enable direct copying of entity attributes from the KG subgraph into generated questions, enhancing factual accuracy and fluency.
• To evaluate the effectiveness of the proposed QG model as a data augmentation technique for knowledge base question answering (KBQA).
• To investigate the impact of node/edge embedding initialization and answer representation on question generation performance.

Proposed method

• A bidirectional Graph2Seq model is used to encode KG subgraphs, capturing structural dependencies through graph neural networks (GNNs) instead of sequential encoders.
• Node-level copying mechanism is introduced to allow direct copying of node attributes (e.g., entity names) from the input KG subgraph into the generated question, improving factual consistency.
• Node and edge embeddings are initialized using pre-trained BERT-based representations and relation-specific embeddings to capture semantic and relational information.
• The model employs an RNN decoder with coverage mechanism and attention to generate questions token-by-token while attending to relevant subgraph components.
• Multi-relational graphs are handled via relation-aware message passing in the GNN encoder, preserving edge-specific semantics.
• Answer information is encoded via markup vectors indicating whether a node is the target answer, guiding question focus.

Experimental results

Research questions

• RQ1 Can a graph-based encoder outperform sequence-based encoders in capturing structural information for multi-hop KG question generation?
• RQ2 Does a node-level copying mechanism improve the factual accuracy and fluency of generated questions by directly incorporating entity attributes?
• RQ3 To what extent can a QG model trained on KG subgraphs improve downstream KBQA performance through data augmentation?
• RQ4 How do different node and edge embedding initialization strategies affect the quality of generated questions?
• RQ5 Can answer-aware representation improve the relevance and focus of generated questions?

Key findings

• The proposed G2S+AE model achieves new state-of-the-art performance on two public KG-QG benchmarks, outperforming existing methods by a significant margin.
• The model’s performance is highly sensitive to node and edge embedding initialization, with BERT-based initialization yielding the best results.
• The node-level copying mechanism improves factual consistency, as evidenced by higher BLEU and ROUGE scores and better human evaluation on entity mention accuracy.
• The model consistently improves KBQA performance, especially when training data is scarce—achieving the largest F1 score boost at 5% and 10% data proportions.
• Data augmentation using the G2S+AE-generated questions improves the F1 score of the BAMnet KBQA baseline more than using questions from a Transformer-based QG model.
• Coverage mechanism did not improve performance, likely due to over-encouragement of irrelevant subgraph tuples, suggesting a need for more selective attention mechanisms.