[Paper Review] Structural Scaffolds for Citation Intent Classification in Scientific Publications

Arman Cohan, Waleed Ammar+2 more|arXiv (Cornell University)|Apr 2, 2019

Topic Modeling4 citations

TL;DR

This paper proposes a neural multitask learning framework that uses structural scaffolds—predicting section titles and citation worthiness—as auxiliary tasks to improve citation intent classification in scientific papers. By leveraging intrinsic structural signals from scientific discourse without external features, the model achieves a new state-of-the-art F1 score of 67.9% on the ACL-ARC dataset, a 13.3% absolute improvement over prior work, and introduces the large-scale SciCite dataset covering multiple scientific domains.

ABSTRACT

Identifying the intent of a citation in scientific papers (e.g., background information, use of methods, comparing results) is critical for machine reading of individual publications and automated analysis of the scientific literature. We propose structural scaffolds, a multitask model to incorporate structural information of scientific papers into citations for effective classification of citation intents. Our model achieves a new state-of-the-art on an existing ACL anthology dataset (ACL-ARC) with a 13.3% absolute increase in F1 score, without relying on external linguistic resources or hand-engineered features as done in existing methods. In addition, we introduce a new dataset of citation intents (SciCite) which is more than five times larger and covers multiple scientific domains compared with existing datasets. Our code and data are available at: https://github.com/allenai/scicite.

Motivation & Objective

• To improve citation intent classification in scientific literature by incorporating structural information from scientific papers.
• To eliminate reliance on hand-engineered linguistic features or external resources in citation intent classification.
• To develop a scalable, general-purpose dataset for citation intent classification across diverse scientific domains.
• To demonstrate that auxiliary tasks related to paper structure can significantly enhance the performance of the main classification task.

Proposed method

• The model uses a hybrid word representation combining GloVe and ELMo embeddings to encode citation contexts.
• A bidirectional LSTM processes the contextualized token representations to capture sequential dependencies.
• An attention mechanism aggregates the LSTM outputs into a single context-aware representation for each citation.
• Two structural scaffolds are introduced: predicting the section title and determining if a sentence needs a citation (citation worthiness).
• The framework employs multitask learning, jointly training the main intent classification task with the two auxiliary scaffold tasks.
• The model is trained end-to-end on a large auxiliary dataset of paper structures, improving generalization for citation intent prediction.

Experimental results

Research questions

• RQ1 Can structural signals inherent in scientific papers improve citation intent classification without relying on external linguistic resources?
• RQ2 How effective are auxiliary tasks like section title prediction and citation worthiness prediction in enhancing the main citation intent classification task?
• RQ3 Can a unified, general-purpose citation intent classification scheme be developed that applies across multiple scientific domains?
• RQ4 To what extent does incorporating structural scaffolds outperform existing feature-based methods in citation intent classification?

Key findings

• The proposed scaffold model achieves a new state-of-the-art F1 score of 67.9% on the ACL-ARC dataset, representing a 13.3% absolute improvement over the previous best method.
• The model outperforms existing feature-based approaches without using any hand-engineered linguistic features or external resources.
• The introduced SciCite dataset is more than five times larger than existing citation intent datasets and spans multiple scientific domains.
• The confusion matrices show that the model generalizes well, with minimal error on major intent categories across both ACL-ARC and SciCite datasets.
• The auxiliary tasks of section title prediction and citation worthiness prediction significantly improve the main intent classification task through multitask learning.
• The model demonstrates strong performance on the new SciCite dataset, validating its generalization capability across diverse scientific literature.