[Paper Review] RH-Net: Improving Neural Relation Extraction via Reinforcement Learning and Hierarchical Relational Searching

Jianing Wang|arXiv (Cornell University)|Oct 27, 2020

Topic Modeling21 references4 citations

TL;DR

RH-Net proposes a novel framework that jointly addresses noisy labeling and long-tail distribution in distant supervision-based relation extraction by integrating reinforcement learning for high-quality instance selection and a hierarchical relational search module to transfer semantics from data-rich to data-poor relations. The method achieves state-of-the-art performance on the NYT dataset through iterative interaction between the two components.

ABSTRACT

Distant supervision (DS) aims to generate large-scale heuristic labeling corpus, which is widely used for neural relation extraction currently. However, it heavily suffers from noisy labeling and long-tail distributions problem. Many advanced approaches usually separately address two problems, which ignore their mutual interactions. In this paper, we propose a novel framework named RH-Net, which utilizes Reinforcement learning and Hierarchical relational searching module to improve relation extraction. We leverage reinforcement learning to instruct the model to select high-quality instances. We then propose the hierarchical relational searching module to share the semantics from correlative instances between data-rich and data-poor classes. During the iterative process, the two modules keep interacting to alleviate the noisy and long-tail problem simultaneously. Extensive experiments on widely used NYT data set clearly show that our method significant improvements over state-of-the-art baselines.

Motivation & Objective

• To address the dual challenges of noisy labeling and long-tail distribution in distant supervision-based relation extraction.
• To jointly optimize noisy instance filtering and semantics transfer from data-rich to data-poor relations.
• To improve model generalization and performance on low-resource relation classes.
• To develop a tree-search-based hierarchical relational search mechanism that shares correlated semantics across relation types.
• To design a reinforcement learning agent enhanced with implicit relation information for better instance selection.

Proposed method

• Reinforcement learning is used to train an agent that splits relation bags into correct and noisy sets, selecting only high-quality instances for downstream training.
• The hierarchical relational search module treats relation extraction as a tree search task, traversing a predefined relation hierarchy from root to leaf nodes.
• A gating mechanism combines semantics from related instances at each node, enabling knowledge transfer from data-rich to data-poor relations.
• A weighted sum loss function is introduced to balance contributions across different layers of the hierarchical tree, improving training stability.
• Implicit relation information is integrated into the reward function of the RL agent to enhance its ability to detect semantically relevant instances.
• The two modules—reinforcement learning and hierarchical search—interact iteratively during training to jointly alleviate noise and long-tail issues.

Experimental results

Research questions

• RQ1 Can reinforcement learning with implicit relation supervision improve the selection of high-quality training instances in distant supervision?
• RQ2 Can hierarchical relational search effectively transfer semantics from data-rich to data-poor relations in a tree-structured relation hierarchy?
• RQ3 Does the interaction between noisy instance filtering and semantics sharing lead to better performance on long-tail relations?
• RQ4 How does the integration of a gating mechanism and weighted loss function affect model stability and performance?
• RQ5 To what extent does RH-Net outperform existing SOTA methods on the NYT benchmark?

Key findings

• RH-Net achieves state-of-the-art performance on the NYT dataset, outperforming existing SOTA methods in both overall and long-tail relation extraction.
• The PCNN-based RH-Net model achieves higher performance than the CNN-based variant, indicating the importance of piecewise feature representation for long-tail relations.
• The HRS module significantly improves performance over simple attention mechanisms, with a 10%+ drop in P@N when the gating mechanism is removed.
• Removing the implicit relation signal in the RL agent reduces average P@N by 2.10%, showing its value in identifying relevant instances.
• The weighted loss function contributes substantially to training stability, as its removal causes a 7.30% drop in average P@N.
• Case studies confirm that the model successfully identifies correct instances for long-tail relations and filters out noisy sentences, though performance drops on extremely low-resource relations due to insufficient data.