[Paper Review] Graph Convolution over Pruned Dependency Trees Improves Relation Extraction

Yuhao Zhang, Peng Qi+1 more|arXiv (Cornell University)|Sep 26, 2018

Topic Modeling40 citations

TL;DR

This paper extends graph convolutional networks to encode dependency structures for relation extraction, introduces a path-centric pruning strategy, and achieves state-of-the-art results on TACRED with notable speedups over tree-based models. Contextualization with BiLSTM further boosts performance and shows complementary strengths to sequence models.

ABSTRACT

Dependency trees help relation extraction models capture long-range relations between words. However, existing dependency-based models either neglect crucial information (e.g., negation) by pruning the dependency trees too aggressively, or are computationally inefficient because it is difficult to parallelize over different tree structures. We propose an extension of graph convolutional networks that is tailored for relation extraction, which pools information over arbitrary dependency structures efficiently in parallel. To incorporate relevant information while maximally removing irrelevant content, we further apply a novel pruning strategy to the input trees by keeping words immediately around the shortest path between the two entities among which a relation might hold. The resulting model achieves state-of-the-art performance on the large-scale TACRED dataset, outperforming existing sequence and dependency-based neural models. We also show through detailed analysis that this model has complementary strengths to sequence models, and combining them further improves the state of the art.

Motivation & Objective

• Motivate and improve relation extraction by leveraging dependency trees without losing crucial information like negation.
• Develop a GCN-based architecture that efficiently pools information over arbitrary dependency structures.
• Introduce path-centric pruning to remove irrelevant content while preserving informative off-path information.
• Enhance robustness by adding contextualization through BiLSTM before GCN processing.
• Demonstrate state-of-the-art results on TACRED and SemEval, and analyze complementary strengths with sequence models.

Proposed method

• Extend graph convolutional networks to operate on dependency graphs with self-loops and normalized message passing.
• Encode sentences with GCN layers to obtain word representations, then pool to sentence, and concatenate with entity representations for classification.
• Add a contextualized GCN (C-GCN) by feeding input through a BiLSTM before the GCN.
• Introduce path-centric pruning: keep nodes within distance K from the shortest dependency path between entities, with K in {0,1,2,∞} and find K=1 yields best performance.
• Optionally combine GCN/C-GCN with a strong sequence model (PA-LSTM) via probability interpolation to improve performance.
• Compare against dependency-based and sequence models on TACRED and SemEval datasets, reporting F1, precision, and recall.

Experimental results

Research questions

• RQ1 Can a graph convolutional approach over dependency trees outperform traditional dependency-based neural models for relation extraction?
• RQ2 Does incorporating off-path information via pruning improve robustness and accuracy without sacrificing essential content?
• RQ3 Does contextualizing GCN with BiLSTM improve relation extraction performance and robustness to parse errors?
• RQ4 Are dependency-based models complementary to sequence-based models, and can their combination yield state-of-the-art results?

Key findings

• GCN-based relation extraction outperforms prior dependency-based neural models on TACRED and SemEval.
• Contextualized GCN (C-GCN) further improves TACRED performance over GCN, achieving state-of-the-art results.
• Path-centric pruning with K=1 provides the best balance, improving performance by including informative off-path content while removing noise.
• Combining GCN/C-GCN with PA-LSTM yields higher F1 than either model alone, demonstrating complementary strengths.
• On TACRED, GCN and C-GCN achieve F1 boosts over baselines, with C-GCN reaching 66.4 (single) and 68.2 (with PA-LSTM interpolation).
• On SemEval, C-GCN achieves 84.8 F1 with-m and 76.5 F1 mask-m, surpassing several dependency-based models.