[Paper Review] U-Time: A Fully Convolutional Network for Time Series Segmentation Applied to Sleep Staging

Mathias Perslev, Michael Hejselbak Jensen+3 more|arXiv (Cornell University)|Oct 24, 2019

EEG and Brain-Computer InterfacesNeuroscience36 references130 citations

TL;DR

U-Time introduces a fully convolutional encoder–decoder (U-Net style) for time-series segmentation, achieving sleep staging across diverse EEG datasets without task-specific tuning and often surpassing CNN-LSTM baselines.

ABSTRACT

Neural networks are becoming more and more popular for the analysis of physiological time-series. The most successful deep learning systems in this domain combine convolutional and recurrent layers to extract useful features to model temporal relations. Unfortunately, these recurrent models are difficult to tune and optimize. In our experience, they often require task-specific modifications, which makes them challenging to use for non-experts. We propose U-Time, a fully feed-forward deep learning approach to physiological time series segmentation developed for the analysis of sleep data. U-Time is a temporal fully convolutional network based on the U-Net architecture that was originally proposed for image segmentation. U-Time maps sequential inputs of arbitrary length to sequences of class labels on a freely chosen temporal scale. This is done by implicitly classifying every individual time-point of the input signal and aggregating these classifications over fixed intervals to form the final predictions. We evaluated U-Time for sleep stage classification on a large collection of sleep electroencephalography (EEG) datasets. In all cases, we found that U-Time reaches or outperforms current state-of-the-art deep learning models while being much more robust in the training process and without requiring architecture or hyperparameter adaptation across tasks.

Motivation & Objective

• Motivate a robust, plug-and-play time-series segmentation model for physiological signals without recurrent architectures.
• Propose U-Time, a fully feed-forward encoder–decoder (U-Net) tailored for 1D time series like EEG/PSG.
• Demonstrate that a single fixed architecture generalizes across multiple sleep-study datasets.
• Show that dense per-time-point classifications can be aggregated to fix-interval sleep stages without architecture changes.
• Compare to CNN-LSTM baselines and existing methods to establish performance and robustness advantages.

Proposed method

• Develop a fully convolutional 1D encoder–decoder network inspired by U-Net for time-series segmentation.
• Process entire sequences in a single forward pass to output dense per-sample class scores.
• Use a segment classifier that aggregates dense scores over fixed intervals to produce final sleep-stage predictions.
• Employ four encoder blocks with dilated convolutions and aggressive down-sampling to achieve a large receptive field.
• Use four decoder blocks with transposed convolutions and skip connections to recover temporal resolution.
• Train with a generalized dice loss to handle class imbalance and a subject-wise cross-validation scheme.

Experimental results

Research questions

• RQ1 Can a fully convolutional, feed-forward architecture achieve state-of-the-art sleep staging without architecture or hyperparameter tuning for each dataset?
• RQ2 How does U-Time perform across diverse EEG sleep datasets (healthy and diseased) compared to CNN–LSTM baselines and published methods when using a single fixed architecture?
• RQ3 Does the model provide robust, high-resolution per-time-point sleep-stage scores that can be aggregated to standard 30-second labels across datasets?

Key findings

• U-Time achieves equal or superior performance to state-of-the-art methods across seven sleep EEG datasets.
• Across datasets, U-Time matches or outperforms the CNN–LSTM baseline with a fixed architecture and hyperparameters.
• U-Time remains robust to dataset variability and does not require task-specific architectural tuning.
• A single-channel EEG input suffices to reach competitive results, with multi-channel input (e.g., including EOG) improving REM classification in some cases.
• The model outputs dense time-point scores that can be aggregated to 30-second sleep stages, enabling flexible temporal resolution at inference.
• U-Time is computationally efficient, enabling rapid scoring of full PSGs in practice.