[Paper Review] A Unified SPD Token Transformer Framework for EEG Classification: Systematic Comparison of Geometric Embeddings

Chi-Sheng Chen, En-Jui Kuo+3 more|arXiv (Cornell University)|Jan 29, 2026

EEG and Brain-Computer Interfaces0 citations

TL;DR

The paper introduces a unified SPD Token Transformer framework that compares BWSPD, Log-Euclidean, and Euclidean embeddings for EEG covariance matrices, validates theoretical predictions about gradient conditioning and normalization, and achieves state-of-the-art results with multi-band tokenization across motor imagery, ERP, and SSVEP datasets.

ABSTRACT

Spatial covariance matrices of EEG signals are Symmetric Positive Definite (SPD) and lie on a Riemannian manifold, yet the theoretical connection between embedding geometry and optimization dynamics remains unexplored. We provide a formal analysis linking embedding choice to gradient conditioning and numerical stability for SPD manifolds, establishing three theoretical results: (1) BWSPD's $\sqrtκ$ gradient conditioning (vs $κ$ for Log-Euclidean) via Daleckii-Kre\uın matrices provides better gradient conditioning on high-dimensional inputs ($d \geq 22$), with this advantage reducing on low-dimensional inputs ($d \leq 8$) where eigendecomposition overhead dominates; (2) Embedding-Space Batch Normalization (BN-Embed) approximates Riemannian normalization up to $O(\varepsilon^2)$ error, yielding $+26\%$ accuracy on 56-channel ERP data but negligible effect on 8-channel SSVEP data, matching the channel-count-dependent prediction; (3) bi-Lipschitz bounds prove BWSPD tokens preserve manifold distances with distortion governed solely by the condition ratio $κ$. We validate these predictions via a unified Transformer framework comparing BWSPD, Log-Euclidean, and Euclidean embeddings within identical architecture across 1,500+ runs on three EEG paradigms (motor imagery, ERP, SSVEP; 36 subjects). Our Log-Euclidean Transformer achieves state-of-the-art performance on all datasets, substantially outperforming classical Riemannian classifiers and recent SPD baselines, while BWSPD offers competitive accuracy with similar training time.

Motivation & Objective

• Motivate and analyze how embedding geometry on SPD manifolds affects EEG classifier optimization and performance.
• Develop a controlled Transformer framework to compare BWSPD, Log-Euclidean, and Euclidean embeddings within identical architecture.
• Theoretically connect embedding choice to gradient conditioning, normalization, and distance preservation on SPD manifolds.
• Empirically validate predictions across multiple EEG paradigms with extensive training runs and subjects.
• Demonstrate state-of-the-art performance and practical benefits of multi-band tokenization.

Proposed method

• Formulate a unified SPD Token Transformer that operates on SPD covariance matrices.
• Implement three embedding tokens: BWSPD, Log-Euclidean, and Euclidean; share projection, Transformer encoder, and classifier across embeddings.
• Apply Embedding-Space Batch Normalization (BN-Embed) to stabilize training and approximate Riemannian normalization.
• Use a Transformer backbone with identical hyperparameters to ensure fair embedding comparisons.
• Extract upper-triangular entries to form token vectors of size D_token = d(d+1)/2 for each embedding.
• Explore multi-band tokenization (T=3) to model frequency-band information across bands and assess variance reduction.

Experimental results

Research questions

• RQ1 How do different SPD embeddings (BWSPD, Log-Euclidean, Euclidean) affect gradient conditioning and optimization dynamics in EEG classification?
• RQ2 Can BN-Embed approximate Riemannian normalization effectively in high-dimensional embedding spaces?
• RQ3 Do BWSPD, Log-Euclidean, and Euclidean embeddings preserve SPD manifold distances in token space with predictable distortion?
• RQ4 What is the impact of multi-band tokenization on accuracy and variance across EEG datasets?
• RQ5 What are the dataset- and channel-count-dependent effects of embedding choices on performance?

Key findings

• Log-Euclidean embedding achieves state-of-the-art accuracy on all three datasets (BCI2a: 95.37%, BCIcha: 95.21%, MAMEM: 99.07%).
• BWSPD shows competitive accuracy with similar training time, particularly in high-dimensional settings, due to better gradient conditioning.
• BN-Embed provides substantial accuracy gains on high-channel data (BCIcha +26%, BCI2a +23%) and negligible effect on low-channel data (MAMEM).
• Multi-band tokenization (T=3) yields substantial accuracy improvements across datasets (BCI2a +3.96pp, BCIcha +4.24pp, MAMEM +0.90pp) with greatly reduced variance.
• Log-Euclidean often excels in multi-class/ frequency-localized scenarios, while BWSPD benefits optimization conditioning in high-dimensional inputs, aligning with theoretical predictions.
• The framework enables fair, controlled comparisons and demonstrates the sequence modeling capacity of Transformers for SPD-based EEG classification.