[Paper Review] BrainRVQ: A High-Fidelity EEG Foundation Model via Dual-Domain Residual Quantization and Hierarchical Autoregression

Mingzhe Cui, Tao Chen+5 more|arXiv (Cornell University)|Feb 18, 2026

EEG and Brain-Computer Interfaces0 citations

TL;DR

BrainRVQ introduces a dual-domain residual vector quantization tokenizer and hierarchical autoregressive pre-training with an importance-guided curriculum masking to learn high-fidelity EEG representations, achieving state-of-the-art results across 8 downstream EEG tasks.

ABSTRACT

Developing foundation models for electroencephalography (EEG) remains challenging due to the signal's low signal-to-noise ratio and complex spectro-temporal non-stationarity. Existing approaches often overlook the hierarchical latent structure inherent in neural dynamics, leading to suboptimal reconstruction of fine-grained information. In this work, we propose BrainRVQ, a general-purpose EEG foundation model pre-trained on a large-scale corpus of clinical EEG data. Unlike standard masked modeling, BrainRVQ features a Dual-Domain Residual Vector Quantization (DD-RVQ) tokenizer that disentangles temporal waveforms and spectral patterns into hierarchical discrete codes. We further introduce a hierarchical autoregressive pre-training objective that learns to reconstruct these codes in a coarse-to-fine manner, utilizing an importance-guided curriculum masking strategy to prioritize information-rich neural events over background noise. Extensive experiments across 8 diverse downstream datasets demonstrate that BrainRVQ consistently outperforms state-of-the-art baselines, validating its effectiveness in learning robust and generalizable neural representations. Our code and model weights are available:https://github.com/keqicmz/BrainRVQ

Motivation & Objective

• Motivate the need for high-fidelity EEG foundation models due to low SNR and non-stationarity of EEG signals.
• Propose DD-RVQ to jointly encode time-domain and frequency-domain information for richer representations.
• Introduce hierarchical autoregressive pre-training with teacher forcing and an importance-guided curriculum masking.
• Pre-train on a large clinical EEG corpus and validate generalization across diverse downstream tasks.
• Demonstrate superior performance compared to state-of-the-art EEG baselines on multiple benchmarks.

Proposed method

• Dual-Domain Residual Vector Quantization (DD-RVQ) tokenization that produces hierarchical codes in both time and frequency domains.
• Shared encoder with temporal and frequency RVQ branches and domain-specific decoders for waveform and spectral reconstruction.
• Hierarchical Autoregressive Pre-training that models coarse-to-fine dependencies with teacher forcing.
• Importance-Guided Curriculum Masking that prioritizes information-rich neural events via physiology-aware scoring and curriculum scheduling.
• Pre-training on the Temple University Hospital EEG Corpus (TUEG) with a 12-layer Transformer encoder and RVQ codebooks; downstream adapters via fine-tuning on eight EEG datasets.

Experimental results

Research questions

• RQ1 Can a dual-domain (time and frequency) tokenization improve EEG representation fidelity over single-domain approaches?
• RQ2 Does hierarchical residual quantization with autoregressive learning yield better downstream performance than flat or single-layer tokenization?
• RQ3 Can an importance-guided curriculum masking strategy improve learning efficiency and transfer to diverse EEG tasks?
• RQ4 How well does BrainRVQ generalize across seizure detection, emotion recognition, sleep staging, and motor imagery tasks?

Key findings

• BrainRVQ consistently outperforms state-of-the-art baselines on eight downstream EEG datasets.
• On representative tasks, BrainRVQ achieves the highest scores across multiple metrics, including AUROC, AUC-PR, and balanced accuracy.
• Ablations show dual-domain tokenization, hierarchical autoregression, and the importance-guided masking all contribute to performance gains.
• The model exhibits strong performance in seizure detection, mental workload assessment, and motor imagery classification.
• Hierarchical residual quantization provides superior representational granularity, especially for fine-grained motor imagery signals.