[Paper Review] Natural language processing to identify lupus nephritis phenotype in electronic health records

Yu Deng, Jennifer A. Pacheco+15 more|arXiv (Cornell University)|Dec 20, 2021

Hepatitis C virus research4 citations

TL;DR

This study develops and validates NLP-based algorithms to identify lupus nephritis phenotypes in electronic health records (EHRs), integrating structured data and clinical text from pathology reports and notes. The best-performing model—using a mixture of CUIs, regular expressions, and feature counts—achieved an F-measure of 0.96 on external validation at Vanderbilt University Medical Center, significantly outperforming a baseline rule-based approach (F-measure 0.62).

ABSTRACT

Systemic lupus erythematosus (SLE) is a rare autoimmune disorder characterized by an unpredictable course of flares and remission with diverse manifestations. Lupus nephritis, one of the major disease manifestations of SLE for organ damage and mortality, is a key component of lupus classification criteria. Accurately identifying lupus nephritis in electronic health records (EHRs) would therefore benefit large cohort observational studies and clinical trials where characterization of the patient population is critical for recruitment, study design, and analysis. Lupus nephritis can be recognized through procedure codes and structured data, such as laboratory tests. However, other critical information documenting lupus nephritis, such as histologic reports from kidney biopsies and prior medical history narratives, require sophisticated text processing to mine information from pathology reports and clinical notes. In this study, we developed algorithms to identify lupus nephritis with and without natural language processing (NLP) using EHR data. We developed four algorithms: a rule-based algorithm using only structured data (baseline algorithm) and three algorithms using different NLP models. The three NLP models are based on regularized logistic regression and use different sets of features including positive mention of concept unique identifiers (CUIs), number of appearances of CUIs, and a mixture of three components respectively. The baseline algorithm and the best performed NLP algorithm were external validated on a dataset from Vanderbilt University Medical Center (VUMC). Our best performing NLP model incorporating features from both structured data, regular expression concepts, and mapped CUIs improved F measure in both the NMEDW (0.41 vs 0.79) and VUMC (0.62 vs 0.96) datasets compared to the baseline lupus nephritis algorithm.

Motivation & Objective

• To improve identification of lupus nephritis phenotypes in EHRs using natural language processing.
• To overcome limitations of relying solely on structured data and procedure codes in EHRs.
• To integrate unstructured clinical text—especially biopsy reports and narrative notes—into lupus nephritis phenotyping.
• To validate the performance of NLP-enhanced algorithms on an external dataset from Vanderbilt University Medical Center.
• To support large-scale observational studies and clinical trials by enabling accurate, scalable phenotyping of lupus nephritis in EHRs.

Proposed method

• Developed a baseline rule-based algorithm using only structured EHR data (e.g., procedure codes, lab results).
• Built three NLP models using regularized logistic regression with distinct feature sets: (1) positive mention of CUIs, (2) frequency of CUIs, and (3) a hybrid of CUI mentions, frequencies, and regular expression matches.
• Mapped clinical concepts in narrative text to Unified Medical Language System (UMLS) concept unique identifiers (CUIs) for standardization.
• Applied regular expressions to detect key lupus nephritis-related terms in clinical notes and pathology reports.
• Trained and tuned logistic regression models on labeled EHR data to classify patients as having lupus nephritis or not.
• Validated the best-performing NLP model on an independent dataset from Vanderbilt University Medical Center (VUMC) to assess generalizability.

Experimental results

Research questions

• RQ1 Can NLP techniques improve the accuracy of lupus nephritis phenotyping in EHRs compared to rule-based methods relying on structured data alone?
• RQ2 Which combination of NLP features—CUI mentions, CUI frequency, or hybrid features—yields the highest performance in detecting lupus nephritis?
• RQ3 How does the performance of the best NLP model generalize to an external, independent EHR dataset from Vanderbilt University Medical Center?
• RQ4 To what extent do unstructured clinical notes and pathology reports contribute to accurate lupus nephritis identification beyond structured data?
• RQ5 Can a hybrid NLP model incorporating both linguistic patterns and standardized medical concepts (CUIs) outperform models using only one type of feature?

Key findings

• The best-performing NLP model, which combined CUI mentions, CUI frequencies, and regular expression matches, achieved an F-measure of 0.96 on the Vanderbilt University Medical Center (VUMC) external validation dataset.
• This NLP model significantly outperformed the baseline rule-based algorithm, which achieved an F-measure of 0.62 on the same VUMC dataset.
• On the NMEDW dataset, the best NLP model achieved an F-measure of 0.79, compared to 0.41 for the baseline algorithm.
• The inclusion of both structured data and NLP-extracted features from clinical notes and pathology reports substantially improved phenotyping performance.
• The use of CUIs and regular expressions enabled effective extraction of lupus nephritis-related concepts from unstructured text.
• The results demonstrate that NLP can effectively bridge the gap in phenotyping when critical diagnostic information is embedded in narrative EHR content.