[Paper Review] Entity Concept-enhanced Few-shot Relation Extraction

Shan Yang, Yongfei Zhang+3 more|arXiv (Cornell University)|Jun 4, 2021

Topic Modeling18 references4 citations

TL;DR

This paper proposes ConceptFERE, a novel few-shot relation extraction framework that enhances relation classification by integrating entity concepts into the attention and fusion mechanisms. By selecting the most relevant concept per entity via semantic similarity and fusing sentence and concept representations at the word level using self-attention, ConceptFERE achieves state-of-the-art performance on FewRel, outperforming prior methods by up to 4.45% under 5-way 1-shot settings.

ABSTRACT

Few-shot relation extraction (FSRE) is of great importance in long-tail distribution problem, especially in special domain with low-resource data. Most existing FSRE algorithms fail to accurately classify the relations merely based on the information of the sentences together with the recognized entity pairs, due to limited samples and lack of knowledge. To address this problem, in this paper, we proposed a novel entity CONCEPT-enhanced FEw-shot Relation Extraction scheme (ConceptFERE), which introduces the inherent concepts of entities to provide clues for relation prediction and boost the relations classification performance. Firstly, a concept-sentence attention module is developed to select the most appropriate concept from multiple concepts of each entity by calculating the semantic similarity between sentences and concepts. Secondly, a self-attention based fusion module is presented to bridge the gap of concept embedding and sentence embedding from different semantic spaces. Extensive experiments on the FSRE benchmark dataset FewRel have demonstrated the effectiveness and the superiority of the proposed ConceptFERE scheme as compared to the state-of-the-art baselines. Code is available at https://github.com/LittleGuoKe/ConceptFERE.

Motivation & Objective

• To address the challenge of low-resource, long-tail relation extraction in specialized domains where labeled data is scarce.
• To overcome the limitations of existing few-shot relation extraction models that rely solely on sentence and entity pair information without external knowledge.
• To improve relation classification by incorporating concise, high-level entity concepts as external knowledge.
• To design a mechanism that selects the most relevant concept from multiple candidate concepts per entity based on semantic relevance to the sentence.
• To bridge the semantic gap between sentence embeddings and concept embeddings through word-level attention-based fusion.

Proposed method

• A concept-sentence attention module computes semantic similarity between the input sentence and multiple candidate concepts per entity to select the most relevant concept.
• A self-attention based fusion module performs word-level semantic integration between the sentence and the selected concept embedding.
• The framework is built on top of BERT-PAIR, leveraging its metric learning architecture for few-shot classification.
• Concept embeddings are pre-trained and sourced from ConceptNet and Concept Graph, providing abstract, generalizable knowledge.
• The model uses a concatenation of the sentence and selected concept as input to the BERT-based encoder for end-to-end training.
• Ablation studies validate the necessity of both the concept selection and fusion modules for optimal performance.

Experimental results

Research questions

• RQ1 Can entity concepts provide meaningful, discriminative signals for few-shot relation classification beyond raw sentence and entity pair information?
• RQ2 How can multiple, potentially conflicting concepts associated with a single entity be filtered to select the most relevant one for a given relation prediction task?
• RQ3 Does word-level fusion of sentence and concept representations improve generalization in low-resource few-shot scenarios?
• RQ4 How does the integration of concise concepts compare to using long text descriptions from knowledge bases like Wikidata in few-shot relation extraction?
• RQ5 To what extent do concept-based enhancements improve performance over state-of-the-art models that use only plain text or entity descriptions?

Key findings

• ConceptFERE achieves a 5-way 1-shot accuracy of 89.21% on the FewRel benchmark, outperforming the previous SOTA method TD-Proto by 4.45 percentage points.
• The model improves upon BERT-PAIR, the strongest baseline without external knowledge, by 6.64 percentage points under the 5-way 1-shot setting.
• Ablation studies confirm that both the concept-sentence attention and the fusion module are essential, as removing either reduces performance by over 6 percentage points.
• The simplified version of ConceptFERE, which omits attention and fusion modules, still outperforms BERT-PAIR, demonstrating the value of concept integration alone.
• The performance gain is consistent across settings, indicating that concept-based knowledge enhances generalization in low-shot scenarios.
• The results suggest that concept-level knowledge is more effective and robust than long-form text descriptions for few-shot relation extraction.