[Paper Review] Multi-Task Identification of Entities, Relations, and Coreference for Scientific Knowledge Graph Construction

Yi Luan, Luheng He+2 more|arXiv (Cornell University)|Aug 29, 2018

Topic Modeling36 references44 citations

TL;DR

The paper introduces a unified multi-task model (SciIE) for identifying scientific entities, relations, and coreference, along with a SciERC dataset, and demonstrates improved information extraction and knowledge graph construction without hand-engineered features.

ABSTRACT

We introduce a multi-task setup of identifying and classifying entities, relations, and coreference clusters in scientific articles. We create SciERC, a dataset that includes annotations for all three tasks and develop a unified framework called Scientific Information Extractor (SciIE) for with shared span representations. The multi-task setup reduces cascading errors between tasks and leverages cross-sentence relations through coreference links. Experiments show that our multi-task model outperforms previous models in scientific information extraction without using any domain-specific features. We further show that the framework supports construction of a scientific knowledge graph, which we use to analyze information in scientific literature.

Motivation & Objective

• Motivate advancing scientific information extraction by jointly modeling entities, relations, and coreference to handle cross-sentence information and reduce cascading errors.
• Create SciERC, a dataset with annotations for entities, relations, and coreference in scientific abstracts.
• Develop a unified, end-to-end framework that shares span representations across tasks for improved span boundary detection and relation coverage.
• Demonstrate that the multi-task model improves over state-of-the-art baselines on SciERC and SemEval datasets.
• Show how extracted terms and relations can be integrated into a large-scale scientific knowledge graph for literature analysis.

Proposed method

• Treat three tasks (entity recognition, relation extraction, coreference resolution) as multinomial classifications over shared span representations.
• Enumerate and score candidate spans up to a maximum width, enabling overlap and cross-sentence relations.
• Use a unified scoring architecture with Phi_E, Phi_R, and Phi_C computed from shared span representations g_i via feed-forward networks; relations and coreference use pairwise span features including g_i, g_j, and g_i ∘ g_j.
• Train with a weighted sum of negative log-likelihood losses for E, R, and C, with task weights lambda_E, lambda_R, lambda_C to balance tasks.
• Incorporate beam pruning to reduce candidate spans and pairwise computations, enabling efficient inference; share span representations across tasks to propagate cross-sentence information via coreference.

Experimental results

Research questions

• RQ1 Can a single multi-task model jointly identify entities, relations, and coreference in scientific text outperform task-specific baselines?
• RQ2 Does cross-sentence information via coreference improve relation coverage and entity boundary detection in scientific articles?
• RQ3 Can a span-based end-to-end approach without hand-engineered features effectively handle overlapping spans and cross-sentence relations in scientific IE?
• RQ4 How does joint learning impact the construction and quality of a downstream scientific knowledge graph?
• RQ5 What insights can be drawn from coreference propagation on improving knowledge graph density and disambiguation?

Key findings

• SciIE outperforms baselines on SciERC for entity recognition, relation extraction, and coreference resolution.
• Entity recognition: 68.1 F1 on development and 64.2 F1 on test in SciIE, with notable gains over LSTM+CRF and E2E Rel baselines.
• Relation extraction: SciIE achieves 58.0 F1 (dev) and 48.2 F1 (test), substantially surpassing E2E Rel baselines.
• Coreference resolution: SciIE achieves 58.0 F1 on development and 48.2 F1 on test, outperforming E2E Coref.
• Ablation shows multi-task learning improves performance across tasks; joint training yields gains such as entity 65.7 vs 68.1, relation 37.9 vs 39.5, coreference 55.3 vs 58.0 in development.
• On SemEval 2017, SciIE outperforms previous feature-based methods (Span identification F1 58.6; Keyphrase extraction F1 46.0; Relation extraction F1 27.8; Overall F1 44.7).
• Knowledge graph construction from 110k abstracts demonstrates that coreference propagation increases graph quality and recall in evaluated human-annotated triples.