[Paper Review] Exploring Graph-structured Passage Representation for Multi-hop Reading Comprehension with Graph Neural Networks

Linfeng Song, Zhiguo Wang+4 more|arXiv (Cornell University)|Sep 6, 2018

Topic Modeling33 references86 citations

TL;DR

The paper builds graph-structured representations over passages with three edge types (same-entity, coreference, window) and applies graph neural networks (GRN/GCN) to perform evidence integration for multi-hop reading comprehension, achieving state-of-the-art results on WikiHop and ComplexWebQuestions without extra annotations.

ABSTRACT

Multi-hop reading comprehension focuses on one type of factoid question, where a system needs to properly integrate multiple pieces of evidence to correctly answer a question. Previous work approximates global evidence with local coreference information, encoding coreference chains with DAG-styled GRU layers within a gated-attention reader. However, coreference is limited in providing information for rich inference. We introduce a new method for better connecting global evidence, which forms more complex graphs compared to DAGs. To perform evidence integration on our graphs, we investigate two recent graph neural networks, namely graph convolutional network (GCN) and graph recurrent network (GRN). Experiments on two standard datasets show that richer global information leads to better answers. Our method performs better than all published results on these datasets.

Motivation & Objective

• Motivate and improve multi-hop reading comprehension by connecting global evidence beyond local coreference information.
• Propose a richer graph construction over entity mentions across passages using three edge types.
• Apply graph neural networks (GRN/GCN) to encode the graph and integrate evidence for answer selection.

Proposed method

• Construct graphs from entity mentions and pronouns across passages using three edge types: same-entity, coreference, and window-based edges.
• Encode graphs with Graph Recurrent Network (GRN) or Graph Convolutional Network (GCN) to propagate information.
• Represent each entity mention and question with neural encodings, then perform additive attention to score candidate answers across graph states.
• Aggregate per-mention scores for each candidate and normalize to obtain final probabilities.
• Train end-to-end with cross-entropy loss using Adam optimizer.
• Compare against baselines including Local BiLSTM and Coref-LSTM/DAG-LSTM variants, and perform ablations on edge types.

Experimental results

Research questions

• RQ1 Does enriching global evidence with multiple edge types improve multi-hop reasoning over text passages?
• RQ2 How do GRN and GCN compare in encoding richer graph structures for MHRC?
• RQ3 What is the contribution of each edge type (same, coreference, window) to overall performance?
• RQ4 Can a graph-based evidence integration framework outperform DAG-based coreference approaches on standard MHRC datasets?

Key findings

• MHQA-GRN achieves 65.4% accuracy on the WikiHop test set, the best published result at submission time.
• MHQA-GRN consistently outperforms MHQA-GCN, and both outperform baselines on WikiHop and ComplexWebQuestions.
• Ablation shows all three edge types contribute to performance, with window edges being less impactful than same-coreference relations but still beneficial.
• On WikiHop dev, GRN with three transition steps (T=3) yields best results; more steps can introduce noise and degrade performance.
• On ComplexWebQuestions, MHQA-GRN improves over Local and outperforms the SplitQA approach without extra annotations (dev-set numbers: MHQA-GRN 33.2 vs Local 31.2).
• MHQA-GRN shows substantial gains by integrating evidence across multiple passages and related entities, indicating the value of graph-based global context.