[Paper Review] A Bi-LSTM-RNN Model for Relation Classification Using Low-Cost Sequence Features

Fei Li, Meishan Zhang+3 more|arXiv (Cornell University)|Aug 27, 2016

Topic Modeling28 references20 citations

TL;DR

This paper proposes a Bi-LSTM-RNN model that classifies relations between two entities using only low-cost sequence features—words and POS tags—without dependency parsing. By dividing sentences into five parts (before, entity1, middle, entity2, after context), the model captures contextual representations, achieving competitive performance on SemEval-2010 and BioNLP-ST 2016, with the middle context proving most informative and approximating the shortest dependency path.

ABSTRACT

Relation classification is associated with many potential applications in the artificial intelligence area. Recent approaches usually leverage neural networks based on structure features such as syntactic or dependency features to solve this problem. However, high-cost structure features make such approaches inconvenient to be directly used. In addition, structure features are probably domain-dependent. Therefore, this paper proposes a bi-directional long-short-term-memory recurrent-neural-network (Bi-LSTM-RNN) model based on low-cost sequence features to address relation classification. This model divides a sentence or text segment into five parts, namely two target entities and their three contexts. It learns the representations of entities and their contexts, and uses them to classify relations. We evaluate our model on two standard benchmark datasets in different domains, namely SemEval-2010 Task 8 and BioNLP-ST 2016 Task BB3. In the former dataset, our model achieves comparable performance compared with other models using sequence features. In the latter dataset, our model obtains the third best results compared with other models in the official evaluation. Moreover, we find that the context between two target entities plays the most important role in relation classification. Furthermore, statistic experiments show that the context between two target entities can be used as an approximate replacement of the shortest dependency path when dependency parsing is not used.

Motivation & Objective

• To address the limitations of high-cost, domain-dependent structure features like syntactic or dependency parsing in relation classification.
• To develop a model that relies only on low-cost sequence features—words and part-of-speech tags—to improve portability across domains.
• To investigate the relative contribution of different contextual segments (before, middle, after) to relation classification performance.
• To evaluate whether the middle context can serve as an effective approximation of the shortest dependency path (SDP) when parsing is unavailable.

Proposed method

• The model splits each sentence into five segments: before-context, first entity, middle-context (between entities), second entity, and after-context.
• Bi-directional LSTMs process token representations across all segments to capture long-range dependencies and mitigate vanishing gradients.
• Max-pooling is applied to each segment’s hidden representations to produce fixed-length context vectors.
• The five pooled vectors are concatenated and fed into a softmax layer for final relation classification.
• The model uses only words and POS tags as input features, avoiding expensive syntactic or dependency parsing.
• Statistical analysis compares word overlap between the middle context and the shortest dependency path (SDP) across two benchmark datasets.

Experimental results

Research questions

• RQ1 Can a Bi-LSTM-RNN model achieve competitive relation classification performance using only low-cost sequence features like words and POS tags?
• RQ2 Which contextual segment (before, middle, after) contributes most significantly to relation classification?
• RQ3 To what extent does the middle context between two entities approximate the information captured by the shortest dependency path (SDP)?
• RQ4 Can the middle context effectively replace dependency parsing in relation classification tasks?

Key findings

• On the SemEval-2010 Task 8 dataset, the model achieved performance comparable to models using complex structure features.
• On the BioNLP-ST 2016 Task BB3 dataset, the model ranked third among all official submissions, demonstrating strong generalization.
• The middle context contributed the most to classification accuracy, outperforming before, after, and entity representations.
• In SemEval-2010, 82% of words in the shortest dependency path (SDP) were also present in the middle context.
• In BioNLP-ST 2016, 61% of SDP words were found in the middle context, indicating strong overlap despite longer, noisier biomedical text.
• The results suggest that the middle context can serve as a reliable, low-cost proxy for the SDP when dependency parsing is not feasible.