[Paper Review] NegBERT: A Transfer Learning Approach for Negation Detection and Scope Resolution

Aditya Khandelwal, Suraj Sawant|arXiv (Cornell University)|Nov 11, 2019

Natural Language Processing Techniques39 references49 citations

TL;DR

NegBERT applies transfer learning with BERT to negation detection and scope resolution, achieving state-of-the-art token-level F1 on four benchmarks across BioScope, Sherlock, and SFU datasets.

ABSTRACT

Negation is an important characteristic of language, and a major component of information extraction from text. This subtask is of considerable importance to the biomedical domain. Over the years, multiple approaches have been explored to address this problem: Rule-based systems, Machine Learning classifiers, Conditional Random Field Models, CNNs and more recently BiLSTMs. In this paper, we look at applying Transfer Learning to this problem. First, we extensively review previous literature addressing Negation Detection and Scope Resolution across the 3 datasets that have gained popularity over the years: the BioScope Corpus, the Sherlock dataset, and the SFU Review Corpus. We then explore the decision choices involved with using BERT, a popular transfer learning model, for this task, and report state-of-the-art results for scope resolution across all 3 datasets. Our model, referred to as NegBERT, achieves a token level F1 score on scope resolution of 92.36 on the Sherlock dataset, 95.68 on the BioScope Abstracts subcorpus, 91.24 on the BioScope Full Papers subcorpus, 90.95 on the SFU Review Corpus, outperforming the previous state-of-the-art systems by a significant margin. We also analyze the model's generalizability to datasets on which it is not trained.

Motivation & Objective

• Motivate the importance of negation detection and scope resolution for information extraction, especially in biomedicine.
• Survey existing approaches and datasets for negation (BioScope, Sherlock, SFU Review Corpus).
• Investigate transfer learning with BERT for negation tasks and establish strong baselines.
• Demonstrate state-of-the-art scope-resolution performance across multiple datasets.

Proposed method

• Apply BERT-based transfer learning to negation detection and scope resolution tasks.
• Experiment with decision choices for adapting BERT to the negation task.
• Evaluate token-level scope-resolution performance on four datasets with established metrics.
• Compare against prior state-of-the-art systems to assess improvements.
• Provide analysis of generalization to datasets not seen during training.

Experimental results

Research questions

• RQ1 Can BERT-based transfer learning improve negation detection and scope resolution over prior methods?
• RQ2 What are the best fine-tuning choices for adapting BERT to negation scope tasks?
• RQ3 How does NegBERT perform on multiple corpora (BioScope Abstracts, BioScope Full Papers, Sherlock, SFU) compared to prior methods?
• RQ4 How well does NegBERT generalize to unseen datasets?

Key findings

• NegBERT achieves token-level F1 scores of 92.36 on Sherlock, 95.68 on BioScope Abstracts, 91.24 on BioScope Full Papers, and 90.95 on SFU Review Corpus.
• NegBERT outperforms previous state-of-the-art systems by a significant margin on all reported datasets.
• The model demonstrates strong generalizability across diverse negation corpora.
• The paper provides analysis of transfer-learning choices for applying BERT to negation tasks.
• The findings establish NegBERT as a leading approach for negation scope resolution at the time.