[Paper Review] InSRL: A Multi-view Learning Framework Fusing Multiple Information Sources for Distantly-supervised Relation Extraction

Zhendong Chu, Haiyun Jiang+2 more|arXiv (Cornell University)|Dec 17, 2020

Topic Modeling31 references4 citations

TL;DR

This paper proposes InSRL, an end-to-end multi-view learning framework that fuses distantly supervised sentences, entity descriptions, and multi-grained entity types for relation extraction. By jointly learning single-view and intact space representations with inner- and cross-view attention, InSRL achieves state-of-the-art performance, improving F1 by 4.6% over baselines on the NYT benchmark.

ABSTRACT

Distant supervision makes it possible to automatically label bags of sentences for relation extraction by leveraging knowledge bases, but suffers from the sparse and noisy bag issues. Additional information sources are urgently needed to supplement the training data and overcome these issues. In this paper, we introduce two widely-existing sources in knowledge bases, namely entity descriptions, and multi-grained entity types to enrich the distantly supervised data. We see information sources as multiple views and fusing them to construct an intact space with sufficient information. An end-to-end multi-view learning framework is proposed for relation extraction via Intact Space Representation Learning (InSRL), and the representations of single views are jointly learned simultaneously. Moreover, inner-view and cross-view attention mechanisms are used to highlight important information on different levels on an entity-pair basis. The experimental results on a popular benchmark dataset demonstrate the necessity of additional information sources and the effectiveness of our framework. We will release the implementation of our model and dataset with multiple information sources after the anonymized review phase.

Motivation & Objective

• To address the sparse and noisy bag problems in distantly supervised relation extraction by incorporating additional information sources.
• To overcome the limitations of traditional multi-instance learning, which assumes at least one positive sentence per bag, a condition often violated in real datasets.
• To develop a unified, end-to-end framework that jointly learns representations from multiple views without requiring view sufficiency.
• To demonstrate the effectiveness of integrating entity descriptions and multi-grained types as complementary signals for relation classification.

Proposed method

• Introduces three information sources as views: distantly supervised sentences (v₁), entity descriptions (v₂), and multi-grained entity types (v₃), each treated as a distinct view.
• Proposes an end-to-end framework, InSRL, that jointly learns single-view representations and an intact space representation, avoiding the two-stage training of standard intact space learning.
• Employs inner-view attention to highlight relation-relevant features within each view, improving representation quality per view.
• Applies cross-view attention to dynamically weight the importance of each view during intact space representation learning, emphasizing informative views like entity descriptions.
• Uses a nonlinear transformation (tanh) on the weighted sum of view embeddings to produce the final intact space representation for classification.
• Trains the entire model end-to-end using a single loss function, enabling joint optimization of all components including attention weights and representation layers.

Experimental results

Research questions

• RQ1 Can entity descriptions and multi-grained entity types serve as effective complementary signals to improve relation extraction in distantly supervised settings?
• RQ2 Does an end-to-end multi-view learning framework that fuses multiple information sources outperform traditional multi-instance learning with only sentence-level supervision?
• RQ3 How effective are inner-view and cross-view attention mechanisms in highlighting relevant information across and within views?
• RQ4 Is intact space representation learning more effective than simple averaging or weighted averaging of views for relation classification?
• RQ5 Does the proposed framework remain robust when individual views are sparse or noisy, as is common in real-world distant supervision?

Key findings

• InSRL achieves an F1 score of 0.465 ± 0.013 on the NYT benchmark, representing a 4.6% improvement over the best baseline (MV-ATT) and a 12.8% improvement over single-view supervision using only sentences.
• The inclusion of entity descriptions (v₂) and multi-grained types (v₃) significantly improves performance, with v₂ alone yielding higher F1 (0.314) than v₃ (0.288), indicating richer semantic content in descriptions.
• Removing the inner-view attention mechanism (InSRL w/o RAT) reduces F1 by 0.028, demonstrating its importance in capturing relation-aware features within each view.
• Removing cross-view attention (InSRL-AVG) leads to a 0.015 drop in F1, showing that dynamically weighting views improves representation quality compared to uniform averaging.
• The MV-AVG and MV-ATT baselines, which use simple averaging and rel-attention without intact space learning, underperform InSRL, proving the effectiveness of the intact space learning framework.
• The ablation study confirms that all three views contribute meaningfully, with the best performance achieved only when all sources are fused via the proposed end-to-end InSRL framework.