[Paper Review] Deep brain state classification of MEG data

Ismail Alaoui Abdellaoui, Jesús García Fernández+2 more|arXiv (Cornell University)|Jul 2, 2020

EEG and Brain-Computer Interfaces31 references4 citations

TL;DR

This paper proposes three deep learning architectures—AA-CascadeNet, AA-MultiviewNet, and a compact CNN with attention—for cross-subject multi-class brain state classification using MEG data from the Human Connectome Project. By integrating self-attention and global attention mechanisms into convolutional and LSTM-based models, the approach improves generalization and achieves up to 94% accuracy, demonstrating enhanced performance and robustness across subjects without requiring domain-specific feature engineering.

ABSTRACT

Neuroimaging techniques have shown to be useful when studying the brain's activity. This paper uses Magnetoencephalography (MEG) data, provided by the Human Connectome Project (HCP), in combination with various deep artificial neural network models to perform brain decoding. More specifically, here we investigate to which extent can we infer the task performed by a subject based on its MEG data. Three models based on compact convolution, combined convolutional and long short-term architecture as well as a model based on multi-view learning that aims at fusing the outputs of the two stream networks are proposed and examined. These models exploit the spatio-temporal MEG data for learning new representations that are used to decode the relevant tasks across subjects. In order to realize the most relevant features of the input signals, two attention mechanisms, i.e. self and global attention, are incorporated in all the models. The experimental results of cross subject multi-class classification on the studied MEG dataset show that the inclusion of attention improves the generalization of the models across subjects.

Motivation & Objective

• To develop end-to-end deep learning models that decode brain states from MEG data without relying on domain-specific feature engineering.
• To investigate the effectiveness of attention mechanisms—specifically self-attention and global attention—in improving model generalization across subjects.
• To explore the transferability of EEG-based deep learning architectures to MEG data, despite differences in signal characteristics.
• To evaluate model performance under varying training data sizes and subject counts, assessing robustness and scalability.

Proposed method

• Proposes AA-CascadeNet, a hybrid architecture combining 1D convolutional layers, bidirectional LSTMs, and an attention-augmented convolutional block with multi-head self-attention.
• Introduces AA-MultiviewNet, a multi-stream network that fuses features from two parallel branches (spatial and temporal) using early fusion and attention mechanisms.
• Employs an attention-augmented convolutional module that combines local convolutional feature extraction with global self-attention to capture long-range dependencies in MEG signals.
• Applies both self-attention and global attention mechanisms: self-attention computes query-key-value interactions across time steps, while global attention focuses on important temporal segments in the LSTM output.
• Uses a residual connection and batch normalization in all models to stabilize training and improve gradient flow.
• Trains all models end-to-end using cross-entropy loss and Adam optimization, with early stopping based on validation loss.

Experimental results

Research questions

• RQ1 To what extent can deep neural networks with attention mechanisms improve cross-subject classification of MEG-based brain states?
• RQ2 How does the integration of self-attention and global attention affect model generalization and performance on unseen subjects?
• RQ3 Does the proposed architecture transfer well from EEG-based models to MEG data, despite differences in signal resolution and noise characteristics?
• RQ4 How does the amount of training data (number of subjects) influence model performance and stability?
• RQ5 Which attention mechanism—self-attention or global attention—contributes more significantly to improved classification accuracy and feature interpretability?

Key findings

• The inclusion of both self-attention and global attention mechanisms significantly improved model generalization, with the best-performing model achieving 94% accuracy on the multiview architecture.
• The AA-MultiviewNet with combined attention mechanisms achieved the highest accuracy (94%) and lowest standard deviation (6%), indicating high consistency across training setups.
• Models with attention mechanisms showed improved feature map preservation through deeper layers, maintaining spatial patterns like the diamond-like shape in MEG signals.
• Global attention weights revealed that the network focused more on specific time steps (e.g., inputs 1, 3, 4, 7, 8 in the temporal stream), indicating selective attention to critical signal segments.
• The model without attention produced sparser feature maps as depth increased, while attention-equipped models retained richer, more informative representations.
• Performance gains from attention were most pronounced with larger training sets, suggesting that attention helps the model learn robust, generalizable representations when sufficient data is available.