[Paper Review] NegBio: a high-performance tool for negation and uncertainty detection in radiology reports

Yifan Peng, Xiaosong Wang+4 more|PubMed|Dec 16, 2017

Topic Modeling26 references129 citations

TL;DR

NegBio is a rule-based tool that uses universal dependencies and subgraph matching to detect negation and uncertainty in radiology reports, improving precision and F1 over NegEx across multiple datasets. It supports open-source use and generalizes to other clinical text.

ABSTRACT

https://github.com/ncbi-nlp/NegBio.

Motivation & Objective

• Reduce false positives in information extraction from radiology reports by accurately identifying negated and uncertain findings.
• Leverage universal dependency graphs to define robust syntactic patterns for negation/uncertainty detection.
• Evaluate performance across multiple radiology corpora and general clinical text to assess robustness and generalizability.

Proposed method

• Extract mentions of medical findings with MetaMap to obtain 14 common radiology finding types.
• Construct universal dependency graphs (UDG) for sentences using a BiLLiP-based parser and Stanford converter.
• Define negation/uncertainty patterns on the UDG and perform subgraph matching to determine scope and applicability.
• Use Semgrex to implement dependency-based rules that capture long-distance and graph-based negation/uncertainty patterns.
• Evaluate NegBio in end-to-end extraction vs. gold standards and compare against NegEx across multiple datasets.

Experimental results

Research questions

• RQ1 Can NegBio improve precision and F1 in extracting positively asserted findings by removing negated/uncertain findings?
• RQ2 How does NegBio perform relative to NegEx across radiology-specific corpora and general clinical text?
• RQ3 Does the syntactic, graph-based approach generalize well to different radiology datasets and to non-radiology clinical texts?
• RQ4 What is the impact of including uncertainty detection on overall system performance?

Key findings

• NegBio substantially improves precision and F1 for positive finding extraction on OpenI and ChestX-ray when combined with MetaMap compared to MetaMap alone and MetaMap+NegEx.
• On OpenI, NegBio achieves P=89.8, R=85.0, F=87.3, outperforming NegEx (P=77.2, R=84.6, F=80.7) and MetaMap alone (P=13.8, R=85.7, F=23.8).
• On ChestX-ray, NegBio attains P=94.4, R=94.4, F=94.4, higher than MetaMap+NegEx (P=82.8, R=95.5, F=88.7) and MetaMap alone (P=72.3, R=95.7, F=82.4).
• NegBio also outperforms NegEx in negation detection on BioScope (P=96.1, R=95.7, F=95.9 vs NegEx P=70.6, R=98.7, F=82.3) and PK (P=98.4, R=88.6, F=93.3 vs NegEx P=95.1, R=91.2, F=93.1).
• Uncertainty detection contributes to improved performance, with disabling uncertainty rules reducing F1 by 7.4% (OpenI) and 2.5% (ChestX-ray) relativamente.
• NegBio’s graph-based, pattern-on-UDG approach yields higher precision by avoiding overlong negation scopes typical of regex-based systems, and improves generalizability across datasets.