[论文解读] Improved Relation Classification by Deep Recurrent Neural Networks with Data Augmentation

Yan Xu, Ran Jia+5 more|arXiv (Cornell University)|Jan 14, 2016

Topic Modeling参考文献 16被引用 124

一句话总结

本文提出在最短依存路径上叠加多层 RNNs 以进行关系分类的 DRNN，并使用基于方向性的数据扩增来提升性能，在使用扩增的情况下在 SemEval-2010 Task 8 上达到 86.1% 的 F1。

ABSTRACT

Nowadays, neural networks play an important role in the task of relation classification. By designing different neural architectures, researchers have improved the performance to a large extent in comparison with traditional methods. However, existing neural networks for relation classification are usually of shallow architectures (e.g., one-layer convolutional neural networks or recurrent networks). They may fail to explore the potential representation space in different abstraction levels. In this paper, we propose deep recurrent neural networks (DRNNs) for relation classification to tackle this challenge. Further, we propose a data augmentation method by leveraging the directionality of relations. We evaluated our DRNNs on the SemEval-2010 Task~8, and achieve an F1-score of 86.1%, outperforming previous state-of-the-art recorded results.

研究动机与目标

• Motivate the use of deeper architectures to capture multi-level abstractions for relation classification beyond shallow networks.
• Develop DRNNs that operate on the shortest dependency path to focus on informative syntactic structure.
• Propose a data augmentation strategy leveraging relation directionality to mitigate data sparseness.
• Evaluate DRNNs on SemEval-2010 Task 8 and compare with prior state-of-the-art methods.

提出的方法

• Build DRNNs by stacking multiple RNN layers on the shortest dependency path between two entities.
• Use four information channels (word embeddings, POS embeddings, grammatical relation embeddings, WordNet embeddings) processed through parallel SDP-based RNNs.
• Incorporate cross-layer connections to enhance information propagation and apply max pooling per layer before concatenation and a final softmax for classification.
• Introduce data augmentation by reversing the sub-paths to generate inverse relations, trained with a joint objective that includes both original and inverted samples (plus L2 regularization).
• Train with cross-entropy loss on pooled representations; use dropout and validation-driven depth selection (up to 4 layers shown to be beneficial).

实验结果

研究问题

• RQ1 Can deeper recurrent architectures on the SDP improve relation classification beyond shallow RNN/CNN models?
• RQ2 Does focusing on SDP reduce irrelevant information and enable better abstraction for relation prediction?
• RQ3 Can data augmentation using relation directionality alleviate data sparseness and enable deeper models to improve performance?
• RQ4 How do DRNNs compare to other neural architectures (CNNs, RNNs, hybrid models) on SemEval-2010 Task 8?
• RQ5 What is the impact of model depth on performance and information propagation across layers?

主要发现

模型	特征	F1
DRNNs (no augmentation)	词嵌入+词性嵌入+语法关系嵌入+WordNet 嵌入	84.2
DRNNs (+ data augmentation)	词嵌入+词性嵌入+GR+WordNet 嵌入	86.1
CNN (SDP-based)	词嵌入	84.0

• DRNNs achieve 84.2% F1 without data augmentation at depth 3, and 86.1% F1 with data augmentation at depth 4.
• Data augmentation via reversing directed relations substantially improves performance (especially when avoiding augmentation of the Other class).
• DRNNs with depth up to 4 outperform various prior methods and CNN-based architectures on SemEval-2010 Task 8 (state-of-the-art with augmentation).
• CNNs do not benefit from deeper architectures in this task, while deeper RNNs continue to improve up to four layers with augmentation.
• Without augmentation, DRNNs are competitive (84.2% F1 at depth 3) but data augmentation provides a notable boost to 86.1% F1.
• The model uses multi-channel SDP representations and shows higher-level layers focus on relation-relevant information, as evidenced by pooling analyses.