[Paper Review] Construction of the Literature Graph in Semantic Scholar

Waleed Ammar, Dirk Groeneveld+21 more|arXiv (Cornell University)|May 6, 2018

Topic ModelingComputer Science21 references48 citations

TL;DR

The paper describes a deployed, scalable system that builds a heterogeneous literature graph (over 280M nodes) by extracting and linking metadata, entities, and mentions from scientific papers using NLP methods, enabling advanced queries and discovery. It covers graph structure, metadata extraction (ScienceParse), entity extraction/linking, KB grounding, and challenges with disambiguation and coverage.

ABSTRACT

We describe a deployed scalable system for organizing published scientific literature into a heterogeneous graph to facilitate algorithmic manipulation and discovery. The resulting literature graph consists of more than 280M nodes, representing papers, authors, entities and various interactions between them (e.g., authorships, citations, entity mentions). We reduce literature graph construction into familiar NLP tasks (e.g., entity extraction and linking), point out research challenges due to differences from standard formulations of these tasks, and report empirical results for each task. The methods described in this paper are used to enable semantic features in www.semanticscholar.org

Motivation & Objective

• Motivate algorithmic discovery in scientific literature by organizing papers, authors, entities, and relations into a unified graph.
• Present a scalable production system for populating the literature graph from diverse sources such as publishers and preprints.
• Describe NLP tasks (metadata extraction, entity extraction/linking) adapted to the scholarly domain and report empirical results.
• Identify challenges (author disambiguation, limited KB coverage, figure extraction) and outline future work.

Proposed method

• Represent the literature as a directed property graph with nodes (papers, authors, entities, mentions) and edges (citations, authorships, entity links, mention relations).
• Use NLP tasks such as sequence labeling, entity linking, and relation extraction to populate the graph from PDFs and metadata.
• Develop the ScienceParse system to predict paper title, authors, and references from PDFs using a multi-stage feature-rich RNN/LSTM pipeline.
• Ground entity mentions to knowledge bases (UMLS, DBpedia) and perform entity linking with a neural scoring model over candidate entities, using token-level and contextual features.
• Pool outputs from multiple entity extraction models (statistical, hybrid, off-the-shelf) to improve precision and yield in deployment.
• Address long-tail challenges (author disambiguation, ontology matching, figure extraction, limited KB coverage) and propose distant supervision for expanding coverage.

Experimental results

Research questions

• RQ1 How can a scalable literature graph be constructed from heterogeneous sources to support advanced scholarly queries?
• RQ2 What NLP approaches are effective for extracting paper metadata, entities, and relations in scientific texts?
• RQ3 How can entity linking to knowledge bases be realized in the scientific domain, and how to handle domain-specific coverage gaps?
• RQ4 What are the main challenges in building and maintaining such a large literature graph, and how can they be mitigated?
• RQ5 What is the impact of combining multiple extraction approaches on precision and coverage in practice?

Key findings

• The literature graph comprises over 280 million nodes including papers, authors, and entities, with various interaction edges like citations and author relations.
• ScienceParse extracts title, authors, and references from PDFs with precision around 85–98% depending on field and target (titles, authors, bibliography fields).
• Three entity extraction/linking approaches (statistical, hybrid, off-the-shelf) yield different tradeoffs; statistical methods give highest precision, hybrid yields highest coverage (yield), and off-the-shelf shows competitive precision in CS but lower precision in Bio.
• Evaluation across CS and Biomedical domains shows the hybrid approach achieving the highest yield (e.g., 1990 CS, 3126 Bio) while statistical approaches achieve high precision (e.g., CS 98.4, Bio 94.4) in document-level tests.
• Entity linking models use KB grounding (UMLS, DBpedia) with neural scoring over candidate entities, achieving Bag of Concepts F1 of 84.6 CS and 85.8 Bio on curated datasets.
• The system pools multiple extraction outputs to improve overall coverage and quality in deployment.