[Paper Review] Classifying Relations via Long Short Term Memory Networks along Shortest Dependency Path

Yan Xu, Lili Mou+4 more|arXiv (Cornell University)|Aug 15, 2015

Topic Modeling29 references114 citations

TL;DR

This paper proposes SDP-LSTM, a neural network that classifies relations between entities by processing the shortest dependency path (SDP) between them using multichannel long short-term memory (LSTM) networks. By leveraging word embeddings, POS tags, grammatical relations, and WordNet hypernyms along the SDP, and applying a customized dropout strategy, the model achieves an F₁-score of 83.7% on the SemEval 2010 benchmark, outperforming prior methods in a fair evaluation setting.

ABSTRACT

Relation classification is an important research arena in the field of natural language processing (NLP). In this paper, we present SDP-LSTM, a novel neural network to classify the relation of two entities in a sentence. Our neural architecture leverages the shortest dependency path (SDP) between two entities; multichannel recurrent neural networks, with long short term memory (LSTM) units, pick up heterogeneous information along the SDP. Our proposed model has several distinct features: (1) The shortest dependency paths retain most relevant information (to relation classification), while eliminating irrelevant words in the sentence. (2) The multichannel LSTM networks allow effective information integration from heterogeneous sources over the dependency paths. (3) A customized dropout strategy regularizes the neural network to alleviate overfitting. We test our model on the SemEval 2010 relation classification task, and achieve an $F_1$-score of 83.7\%, higher than competing methods in the literature.

Motivation & Objective

• To improve relation classification in natural language processing by leveraging syntactic structure and linguistic knowledge.
• To address the challenge of noisy and irrelevant sentence content by focusing on the shortest dependency path (SDP) between entities.
• To model directional relations effectively by splitting the SDP into two sub-paths from each entity to their common ancestor.
• To integrate heterogeneous linguistic information—such as POS tags, grammatical relations, and WordNet hypernyms—into a unified deep learning framework.
• To reduce overfitting in neural networks through a customized dropout strategy tailored to the SDP-LSTM architecture.

Proposed method

• The model extracts the shortest dependency path (SDP) between two target entities in a sentence, retaining only the most relevant words for relation classification.
• The SDP is split into two sub-paths: one from each entity to their lowest common ancestor, enabling direction-sensitive processing.
• Multichannel recurrent neural networks based on LSTM units process different linguistic features (word embeddings, POS tags, grammatical relations, WordNet hypernyms) in parallel along each sub-path.
• Features from the two sub-paths are concatenated and fed into a final classifier for relation prediction using softmax with cross-entropy loss.
• A customized dropout strategy is applied during training to regularize the network and mitigate overfitting, particularly on the SDP representations.
• The model is trained end-to-end on the SemEval 2010 dataset using stochastic gradient descent with backpropagation through time.

Experimental results

Research questions

• RQ1 Can the shortest dependency path (SDP) serve as a robust and informative representation for relation classification, effectively filtering out irrelevant sentence words?
• RQ2 Does modeling the SDP as two directionally distinct sub-paths improve the capture of relation directionality compared to treating it as a single sequence?
• RQ3 To what extent do heterogeneous linguistic features—such as POS tags, grammatical relations, and WordNet hypernyms—complement word embeddings in improving classification performance?
• RQ4 How effective is the use of LSTM units in capturing long-range dependencies along the SDP compared to standard RNNs or other sequence models?
• RQ5 Can a customized dropout strategy significantly reduce overfitting in relation classification models trained on limited annotated data?

Key findings

• SDP-LSTM achieves an F₁-score of 83.7% on the SemEval 2010 relation classification benchmark, outperforming all competing methods in a fair evaluation without data cleaning.
• Using only word embeddings along the SDP yields an F₁-score of 82.35%, demonstrating strong baseline performance compared to prior models like CNNs (69.7%) and RNNs (74.9–79.1%).
• Incorporating grammatical relations alone boosts performance to 83.21%, and combining all four channels (word, POS, grammatical relations, WordNet) achieves the highest F₁-score of 83.70%.
• The ablation study confirms that LSTM units are more effective than standard RNNs, as the latter achieve only 82.8% F₁ when used without LSTM gates.
• Processing the entire dependency path (rather than two sub-paths) results in a lower F₁-score of 82.2%, indicating that directional modeling enhances performance.
• The addition of POS tags contributes a modest 0.63% improvement, suggesting that while less informative individually, they still provide complementary linguistic cues.