[Paper Review] EEG-GPT: Exploring Capabilities of Large Language Models for EEG Classification and Interpretation

Jonathan W. Kim, Ahmed Alaa+1 more|arXiv (Cornell University)|Jan 31, 2024

EEG and Brain-Computer Interfaces8 citations

TL;DR

EEG-GPT uses a large language model fine-tuned on verbally described EEG features to classify normal vs abnormal EEG with few-shot learning, and demonstrates step-wise reasoning with EEG tools for transparency.

ABSTRACT

In conventional machine learning (ML) approaches applied to electroencephalography (EEG), this is often a limited focus, isolating specific brain activities occurring across disparate temporal scales (from transient spikes in milliseconds to seizures lasting minutes) and spatial scales (from localized high-frequency oscillations to global sleep activity). This siloed approach limits the development EEG ML models that exhibit multi-scale electrophysiological understanding and classification capabilities. Moreover, typical ML EEG approaches utilize black-box approaches, limiting their interpretability and trustworthiness in clinical contexts. Thus, we propose EEG-GPT, a unifying approach to EEG classification that leverages advances in large language models (LLM). EEG-GPT achieves excellent performance comparable to current state-of-the-art deep learning methods in classifying normal from abnormal EEG in a few-shot learning paradigm utilizing only 2% of training data. Furthermore, it offers the distinct advantages of providing intermediate reasoning steps and coordinating specialist EEG tools across multiple scales in its operation, offering transparent and interpretable step-by-step verification, thereby promoting trustworthiness in clinical contexts.

Motivation & Objective

• Motivate a unifying EEG classification framework that leverages large language models to address multi-scale EEG phenomena.
• Evaluate whether an LLM can perform normal/abnormal EEG classification with few-shot and zero-shot learning.
• Demonstrate the potential of tree-of-thought and tool integration to provide transparent, step-by-step EEG interpretation.

Proposed method

• Split EEG recordings into 20-second epochs and compute per-channel features (90th percentile of voltage amplitudes, standard deviation, kurtosis, alpha:delta, theta:alpha, delta:theta power ratios) for channels Cz, T5, T6, O1, O2.
• Convert feature summaries into verbal prompts and fine-tune the da Vinci GPT-3 base LLM using prompt-completion pairs to predict normal vs abnormal labels.
• Evaluate EEG-GPT in few-shot learning against traditional ML and deep-learning baselines using 2% of training data.
• Assess zero-shot performance with and without in-context learning.
• Explore reasoning capabilities by enabling the LLM to use specialized tools (seizure detector, spike detector, qEEG feature comparison) in a Tree of Thoughts framework.
• Demonstrate examples where the framework uses tools to reach a final classification.

Figure 1: Diagram depicting the process of fine-tuning large language models [ 9 ]

Experimental results

Research questions

• RQ1 Can an LLM, when fine-tuned on verbalized EEG features, match traditional ML methods in normal/abnormal EEG classification with limited training data?
• RQ2 Does EEG-GPT retain strong performance in zero-shot and few-shot settings compared to deep learning models trained on raw EEG data?
• RQ3 Can an LLM coordinate multiple EEG analysis tools in a transparent, step-by-step decision process to classify EEGs?
• RQ4 What are the potential failure modes (e.g., hallucinations) of LLM-driven EEG interpretation, and how can human oversight mitigate them?

Key findings

• EEG-GPT achieves AUROC of 0.86 with 2% of training data, outperforming traditional ML methods under similar data constraints.
• In zero-shot mode, EEG-GPT performs better than chance and reaches AUROC 0.63 with in-context learning.
• With 2% data, EEG-GPT matches a deep-learning method trained on all available data, though some SOTA deep models still outperform it when fully trained.
• The Tree of Thoughts framework enables the LLM to use automated seizure detection, spike detection, and qEEG feature comparison tools to reach a final decision.
• Demonstrations illustrate the framework stopping early when evidence is sufficient (seizure example) or integrating multiple tools to reach a classification (non-seizure abnormal example).
• Compared to DL approaches that use raw EEG, EEG-GPT uses a limited feature set yet shows competitive performance in few-shot settings.

Figure 2: Pipeline for few-shot experiment