[Paper Review] Cross-Sentence N-ary Relation Extraction with Graph LSTMs

Nanyun Peng, Hoifung Poon+3 more|arXiv (Cornell University)|Aug 12, 2017

Topic Modeling47 references134 citations

TL;DR

The paper proposes a graph LSTM framework to extract cross-sentence n-ary relations, showing improved accuracy over baselines and enabling distant supervision in biomedicine. It also demonstrates multi-task learning with sub-relations to boost performance.

ABSTRACT

Past work in relation extraction has focused on binary relations in single sentences. Recent NLP inroads in high-value domains have sparked interest in the more general setting of extracting n-ary relations that span multiple sentences. In this paper, we explore a general relation extraction framework based on graph long short-term memory networks (graph LSTMs) that can be easily extended to cross-sentence n-ary relation extraction. The graph formulation provides a unified way of exploring different LSTM approaches and incorporating various intra-sentential and inter-sentential dependencies, such as sequential, syntactic, and discourse relations. A robust contextual representation is learned for the entities, which serves as input to the relation classifier. This simplifies handling of relations with arbitrary arity, and enables multi-task learning with related relations. We evaluate this framework in two important precision medicine settings, demonstrating its effectiveness with both conventional supervised learning and distant supervision. Cross-sentence extraction produced larger knowledge bases. and multi-task learning significantly improved extraction accuracy. A thorough analysis of various LSTM approaches yielded useful insight the impact of linguistic analysis on extraction accuracy.

Motivation & Objective

• Motivate cross-sentence, n-ary relation extraction in high-value domains like biomedicine.
• Develop a unified graph LSTM framework that encodes intra- and inter-sentential dependencies.
• Learn robust contextual representations for entities to feed a relation classifier.
• Enable multi-task learning with related sub-relations to improve extraction accuracy.
• Evaluate under supervised and distant supervision settings and analyze linguistic contributions.

Proposed method

• Represent text as a document graph capturing word adjacency, syntax, and discourse relations.
• Use graph LSTMs to compute contextual word and entity representations through forward and backward DAG passes.
• Concatenate entity representations to form inputs for relation classifiers for arbitrary arity.
• Explore two parameterizations: full edge-type specific parameters and edge-type embeddings with tensor products.
• Apply multi-task learning by adding classifiers for related sub-relations sharing the same graph LSTM representations.

Experimental results

Research questions

• RQ1 Can graph LSTMs effectively model cross-sentence n-ary relations by leveraging rich intra- and inter-sentential dependencies?
• RQ2 Do edge-type specific parameters or edge-type embeddings improve cross-sentence relation extraction?
• RQ3 Does multi-task learning with sub-relations boost performance for n-ary and binary relations?
• RQ4 What is the impact of syntactic/discourse information in the biomedical domain for cross-sentence extraction?
• RQ5 How does cross-sentence extraction scale with distant supervision in large biomedical corpora?

Key findings

• Graph LSTM-FULL and Graph LSTM-EMBED outperform baselines (feature-based, CNN, BiLSTM) in cross-sentence ternary relation extraction.
• Graph LSTMs significantly outperform BiLSTMs and CNNs with p<0.05 (McNemar’s test) in cross-sentence settings.
• Multi-task learning with sub-relations yields significant gains for both ternary and binary relations.
• Edge-type embedding provides comparable performance to full parametrization, suggesting parameter efficiency.
• Cross-sentence extraction from PubMed Central yields many more candidate and detected interactions than single-sentence extraction.