[论文解读] Learning Relation Prototype from Unlabeled Texts for Long-tail Relation Extraction

Yixin Cao, Jun Kuang+4 more|arXiv (Cornell University)|Nov 27, 2020

Topic Modeling参考文献 50被引用 5

一句话总结

该论文提出了一种原型学习方法，通过从无标签文本中提取关系原型，实现从高频关系到低频关系的知识迁移，从而提升长尾关系抽取的性能。通过构建共现图并建模实体的一阶和二阶邻近性，该方法优化了实体对与关系原型之间的距离，在NYT和Google远监督数据集上相比当前最先进基线方法平均F1提升4.1%。

ABSTRACT

Relation Extraction (RE) is a vital step to complete Knowledge Graph (KG) by extracting entity relations from texts.However, it usually suffers from the long-tail issue. The training data mainly concentrates on a few types of relations, leading to the lackof sufficient annotations for the remaining types of relations. In this paper, we propose a general approach to learn relation prototypesfrom unlabeled texts, to facilitate the long-tail relation extraction by transferring knowledge from the relation types with sufficient trainingdata. We learn relation prototypes as an implicit factor between entities, which reflects the meanings of relations as well as theirproximities for transfer learning. Specifically, we construct a co-occurrence graph from texts, and capture both first-order andsecond-order entity proximities for embedding learning. Based on this, we further optimize the distance from entity pairs tocorresponding prototypes, which can be easily adapted to almost arbitrary RE frameworks. Thus, the learning of infrequent or evenunseen relation types will benefit from semantically proximate relations through pairs of entities and large-scale textual information.We have conducted extensive experiments on two publicly available datasets: New York Times and Google Distant Supervision.Compared with eight state-of-the-art baselines, our proposed model achieves significant improvements (4.1% F1 on average). Furtherresults on long-tail relations demonstrate the effectiveness of the learned relation prototypes. We further conduct an ablation study toinvestigate the impacts of varying components, and apply it to four basic relation extraction models to verify the generalization ability.Finally, we analyze several example cases to give intuitive impressions as qualitative analysis. Our codes will be released later.

研究动机与目标

• 解决关系抽取中的长尾问题，即少样本或未见关系缺乏足够的训练数据。
• 通过仅利用无标签文本，减少对外部先验知识（如关系层级或实体类型）的依赖。
• 通过学习到的语义原型，实现从高资源关系到低资源关系的知识迁移。
• 通过集成基于原型的迁移学习，提升现有关系抽取模型的泛化能力。
• 开发一种与多种关系抽取架构兼容的通用框架，且无需修改网络结构。

提出的方法

• 从无标签文本中构建共现图，以建模跨句子的实体邻近性。
• 捕捉实体之间的**一阶邻近性**（直接共现）和**二阶邻近性**（共享邻居）
• 将关系原型学习为反映关系语义意义和邻近性的潜在表征。
• 使用对比损失目标函数，优化实体对与其对应关系原型之间的距离。
• 通过即插即用机制，将原型学习模块集成到现有关系抽取模型中。
• 利用大规模文本信息，隐式捕捉关系类型之间的语义相似性，而无需显式监督。

实验结果

研究问题

• RQ1 能否从无标签文本中有效学习关系原型，以提升长尾关系抽取性能？
• RQ2 所提出的原型学习方法如何实现从高频关系到低频关系的知识迁移？
• RQ3 该模型在不修改架构的情况下，跨不同关系抽取框架的泛化能力如何？
• RQ4 一阶与二阶邻近性在原型质量及下游性能中分别起到何种作用？
• RQ5 基于原型的知识迁移对罕见和未见关系类型有何影响？

主要发现

• 所提方法在NYT和Google远监督数据集上，相比八个最先进基线方法，平均F1提升4.1%。
• 在长尾关系上观察到显著性能提升，证实了基于原型的知识迁移的有效性。
• 消融实验表明，一阶与二阶邻近性均对性能有贡献，其中二阶邻近性提供了更强的归纳偏置。
• 原型学习模块在四种不同基础关系抽取模型上均表现出良好的泛化能力，证明了其强大的兼容性与可迁移性。
• 定性案例研究证实，语义相似的关系（如/location/us_state/capital与/location/fr_region/capital）能被学习到的原型正确分组。
• 该方法减少了对外部知识（如关系层级）的依赖，同时通过从原始文本中自监督学习原型，仍实现了优异性能。