[Paper Review] Neural Relation Extraction via Inner-Sentence Noise Reduction and Transfer Learning

Tianyi Liu, Xinsong Zhang+2 more|arXiv (Cornell University)|Aug 21, 2018

Topic Modeling23 references9 citations

TL;DR

This paper proposes a novel neural relation extraction framework that reduces inner-sentence noise via Sub-Tree Parse (STP) and enhances robustness using transfer learning from entity classification. By combining STP, entity-wise attention, and parameter transfer learning, the model improves the Precision/Recall (PR) area from 0.35 to 0.39, outperforming state-of-the-art methods on NYT and Freebase datasets.

ABSTRACT

Extracting relations is critical for knowledge base completion and construction in which distant supervised methods are widely used to extract relational facts automatically with the existing knowledge bases. However, the automatically constructed datasets comprise amounts of low-quality sentences containing noisy words, which is neglected by current distant supervised methods resulting in unacceptable precisions. To mitigate this problem, we propose a novel word-level distant supervised approach for relation extraction. We first build Sub-Tree Parse(STP) to remove noisy words that are irrelevant to relations. Then we construct a neural network inputting the sub-tree while applying the entity-wise attention to identify the important semantic features of relational words in each instance. To make our model more robust against noisy words, we initialize our network with a priori knowledge learned from the relevant task of entity classification by transfer learning. We conduct extensive experiments using the corpora of New York Times(NYT) and Freebase. Experiments show that our approach is effective and improves the area of Precision/Recall(PR) from 0.35 to 0.39 over the state-of-the-art work.

Motivation & Objective

• To address low-quality, noisy sentences in distant supervised relation extraction datasets.
• To reduce word-level noise within sentences that hinders relation extraction performance.
• To improve model robustness against noisy training data through better parameter initialization.
• To leverage entity type classification as a source of prior knowledge for initializing relation extraction models.
• To achieve higher precision and recall in relation extraction despite noisy, informal web-crawled sentences.

Proposed method

• Proposes Sub-Tree Parse (STP) to extract a syntactic subtree centered on the lowest common ancestor of two entities, removing irrelevant words.
• Applies entity-wise attention to emphasize relation-relevant semantic features within the parsed subtree.
• Uses transfer learning to initialize the relation extraction model with pre-trained weights from an entity type classification task.
• Employs a bidirectional GRU (BGRU) with word-level attention for sequence encoding and relation classification.
• Combines STP, entity-wise attention, and transfer learning into an end-to-end neural model for improved noise resilience.
• Trains the model on the NYT and Freebase datasets using distant supervision with automatic labeling.

Experimental results

Research questions

• RQ1 Can inner-sentence noise be effectively reduced using syntactic parsing to improve relation extraction accuracy?
• RQ2 Does entity-wise attention enhance the model’s ability to focus on relation-relevant features compared to standard word-level attention?
• RQ3 Can transfer learning from entity type classification improve the robustness of relation extractors on noisy, low-quality datasets?
• RQ4 How does the integration of STP, entity-wise attention, and transfer learning compare to state-of-the-art relation extraction methods?
• RQ5 To what extent does parameter initialization from a related task reduce the negative impact of noisy training data?

Key findings

• The proposed model achieves a Precision/Recall (PR) curve area of 0.392, representing a 6.3% improvement over the best baseline (PCNN+ATT) in P@100.
• The model increases P@100 by 6.3%, P@200 by 7.6%, and P@300 by 7.7% compared to PCNN+ATT, demonstrating consistent gains across evaluation metrics.
• The combination of STP and entity-wise attention reduces inner-sentence noise effectively, improving performance by over 5% compared to baselines.
• Transfer learning improves model robustness, increasing the PR-AUC by 2% over the model without transfer learning (0.383 vs. 0.392).
• The model maintains high precision (above 0.6) at a recall rate of 0.25, indicating strong performance on high-confidence predictions.
• The STP method reduces average sentence length significantly, with 99.4% of sentences in the NYT-10 dataset containing noise, and the method removes approximately 12 noisy words per sentence on average.