[Paper Review] A deep learning architecture for temporal sleep stage classification using multivariate and multimodal time series

Stanislas Chambon, Mathieu Galtier+3 more|arXiv (Cornell University)|Jul 5, 2017

EEG and Brain-Computer Interfaces50 references594 citations

TL;DR

The paper proposes an end-to-end deep learning model that learns from multivariate and multimodal PSG signals (EEG, EOG, EMG) to classify sleep stages with temporal context, achieving state-of-the-art results on public data.

ABSTRACT

Sleep stage classification constitutes an important preliminary exam in the diagnosis of sleep disorders. It is traditionally performed by a sleep expert who assigns to each 30s of signal a sleep stage, based on the visual inspection of signals such as electroencephalograms (EEG), electrooculograms (EOG), electrocardiograms (ECG) and electromyograms (EMG). We introduce here the first deep learning approach for sleep stage classification that learns end-to-end without computing spectrograms or extracting hand-crafted features, that exploits all multivariate and multimodal Polysomnography (PSG) signals (EEG, EMG and EOG), and that can exploit the temporal context of each 30s window of data. For each modality the first layer learns linear spatial filters that exploit the array of sensors to increase the signal-to-noise ratio, and the last layer feeds the learnt representation to a softmax classifier. Our model is compared to alternative automatic approaches based on convolutional networks or decisions trees. Results obtained on 61 publicly available PSG records with up to 20 EEG channels demonstrate that our network architecture yields state-of-the-art performance. Our study reveals a number of insights on the spatio-temporal distribution of the signal of interest: a good trade-off for optimal classification performance measured with balanced accuracy is to use 6 EEG with 2 EOG (left and right) and 3 EMG chin channels. Also exploiting one minute of data before and after each data segment offers the strongest improvement when a limited number of channels is available. As sleep experts, our system exploits the multivariate and multimodal nature of PSG signals in order to deliver state-of-the-art classification performance with a small computational cost.

Motivation & Objective

• Motivate and address automatic sleep stage classification as an imbalanced multi-class prediction problem.
• Propose an end-to-end deep network that learns from raw multivariate PSG signals without hand-crafted features.
• Leverage spatial filtering and temporal context to improve robustness and performance.
• Benchmark against state-of-the-art methods and analyze the impact of sensor configuration and temporal context.

Proposed method

• Introduce an end-to-end deep architecture that learns linear spatial filters to create virtual channels for each modality (EEG/EOG and EMG).
• Use separate pipelines with convolutional blocks, ReLU, and max-pooling to extract spectral features from each modality.
• Combine modality-specific features and feed to a softmax classifier to output sleep-stage probabilities.
• Extend to a time-distributed multivariate network that aggregates features from neighboring 30 s segments to incorporate temporal context.
• Train with balanced mini-batches and Adam optimization, and employ early stopping for robustness.

Experimental results

Research questions

• RQ1 Can end-to-end learning from raw, multivariate PSG signals outperform hand-crafted feature approaches for sleep stage classification?
• RQ2 How does incorporating spatial (multichannel) information affect classification performance and robustness?
• RQ3 What is the impact of temporal context on classification accuracy and hypnogram transitions?
• RQ4 What is the trade-off between channel count, additional modalities, and computational cost?

Key findings

• Multivariate (EEG + EOG) data with EMG in a parallel pipeline yields state-of-the-art performance on 61 public PSG records.
• Adding more well-distributed EEG channels improves balanced accuracy up to a point, with diminishing gains beyond six channels unless additional modalities are used.
• Incorporating EOG and EMG modalities boosts performance, especially with a smaller EEG channel set.
• Temporal context improves accuracy for certain stages (e.g., N1, N2, REM) when spatial context is limited, but excessive temporal context can reduce performance for wake and N3.
• The proposed multivariate, multimodal architecture achieves strong performance with relatively low computational cost compared to baselines.