[Paper Review] Extracting Lifestyle Factors for Alzheimer's Disease from Clinical Notes Using Deep Learning with Weak Supervision

Zitao Shen, Yoonkwon Yi+4 more|arXiv (Cornell University)|Jan 22, 2021

Machine Learning in Healthcare19 references4 citations

TL;DR

This study proposes a weakly supervised deep learning approach using pre-trained BERT models to extract lifestyle factors—such as physical activity and excessive diet—from unstructured Alzheimer’s disease (AD) clinical notes. By leveraging rule-based labeling to generate weak supervision, the method achieves high F1 scores (0.96 for physical activity, 1.00 for excessive diet), demonstrating the feasibility of scalable, accurate NLP for AD risk factor analysis in electronic health records.

ABSTRACT

Since no effective therapies exist for Alzheimer's disease (AD), prevention has become more critical through lifestyle factor changes and interventions. Analyzing electronic health records (EHR) of patients with AD can help us better understand lifestyle's effect on AD. However, lifestyle information is typically stored in clinical narratives. Thus, the objective of the study was to demonstrate the feasibility of natural language processing (NLP) models to classify lifestyle factors (e.g., physical activity and excessive diet) from clinical texts. We automatically generated labels for the training data by using a rule-based NLP algorithm. We conducted weak supervision for pre-trained Bidirectional Encoder Representations from Transformers (BERT) models on the weakly labeled training corpus. These models include the BERT base model, PubMedBERT(abstracts + full text), PubMedBERT(only abstracts), Unified Medical Language System (UMLS) BERT, Bio BERT, and Bio-clinical BERT. We performed two case studies: physical activity and excessive diet, in order to validate the effectiveness of BERT models in classifying lifestyle factors for AD. These models were compared on the developed Gold Standard Corpus (GSC) on the two case studies. The PubmedBERT(Abs) model achieved the best performance for physical activity, with its precision, recall, and F-1 scores of 0.96, 0.96, and 0.96, respectively. Regarding classifying excessive diet, the Bio BERT model showed the highest performance with perfect precision, recall, and F-1 scores. The proposed approach leveraging weak supervision could significantly increase the sample size, which is required for training the deep learning models. The study also demonstrates the effectiveness of BERT models for extracting lifestyle factors for Alzheimer's disease from clinical notes.

Motivation & Objective

• To address the challenge of extracting lifestyle factors from unstructured clinical notes in electronic health records (EHRs) for Alzheimer’s disease (AD) research.
• To reduce the labor-intensive cost of creating hand-labeled datasets by employing weak supervision through rule-based labeling.
• To evaluate the performance of multiple pre-trained BERT models in classifying lifestyle factors relevant to AD risk.
• To enable large-scale, cost-effective analysis of lifestyle factors and their associations with AD using real-world EHR data.
• To support future causal modeling of lifestyle interventions on cognitive outcomes in AD.

Proposed method

• Generated weak labels for training data using a rule-based NLP algorithm to avoid manual annotation.
• Fine-tuned six BERT-based models: BERT base, PubMedBERT (abstracts + full text), PubMedBERT (only abstracts), UMLS BERT, Bio-BERT, and Bio-clinical BERT.
• Applied weak supervision to train the BERT models on a weakly labeled corpus derived from clinical notes.
• Evaluated model performance on a Gold Standard Corpus (GSC) for two case studies: physical activity and excessive diet.
• Used precision, recall, and F1-score as evaluation metrics to compare model effectiveness.
• Addressed class imbalance and model sensitivity by analyzing misclassification patterns and model-specific limitations.

Experimental results

Research questions

• RQ1 Can weak supervision significantly reduce the need for manual annotation while maintaining high performance in classifying lifestyle factors from clinical notes?
• RQ2 Which pre-trained BERT model performs best for extracting physical activity and excessive diet information from unstructured EHR text?
• RQ3 How do variations in pre-training corpora (e.g., PubMed vs. clinical text) affect model performance on lifestyle factor classification?
• RQ4 To what extent do model misclassifications arise from contradictory phrasing or lack of domain-specific knowledge in certain BERT variants?
• RQ5 Can this approach be scaled to extract multiple lifestyle factors simultaneously for synergistic effect analysis in AD risk?

Key findings

• The PubMedBERT(Abs) model achieved the highest performance for physical activity classification with F1-score of 0.96, precision of 0.96, and recall of 0.96.
• The Bio-BERT model achieved perfect performance for excessive diet classification, with precision, recall, and F1-score all at 1.00.
• Model performance varied significantly based on pre-training corpus and data size, with the smaller excessive diet dataset (n=886) showing greater variance across models.
• Misclassifications were primarily due to contradictory phrasing (e.g., 'physically active' vs. 'without aerobic exercise') and poor handling of 'none' or 'normal' cases by domain-specific BERTs.
• The study demonstrated that weak supervision enables scalable model training without manual annotation, significantly increasing usable training data.
• Despite high overall performance, model sensitivity to data size and class imbalance highlights the need for larger, balanced datasets in future work.