[Paper Review] DeepSleepNet-Lite: A Simplified Automatic Sleep Stage Scoring Model with Uncertainty Estimates

Luigi Fiorillo, Paolo Favaro+1 more|arXiv (Cornell University)|Aug 24, 2021

EEG and Brain-Computer Interfaces52 references106 citations

TL;DR

DeepSleepNet-Lite is a lightweight, single-channel EEG-based sleep staging model that processes only 90-second sequences using a simplified feed-forward CNN architecture. It leverages Monte Carlo dropout at inference to estimate prediction uncertainty, improving performance when uncertain predictions are rejected—achieving up to 86.1% accuracy and 0.81 Cohen’s kappa on Sleep-EDF v1-2013.

ABSTRACT

Deep learning is widely used in the most recent automatic sleep scoring algorithms. Its popularity stems from its excellent performance and from its ability to directly process raw signals and to learn feature from the data. Most of the existing scoring algorithms exploit very computationally demanding architectures, due to their high number of training parameters, and process lengthy time sequences in input (up to 12 minutes). Only few of these architectures provide an estimate of the model uncertainty. In this study we propose DeepSleepNet-Lite, a simplified and lightweight scoring architecture, processing only 90-seconds EEG input sequences. We exploit, for the first time in sleep scoring, the Monte Carlo dropout technique to enhance the performance of the architecture and to also detect the uncertain instances. The evaluation is performed on a single-channel EEG Fpz-Cz from the open source Sleep-EDF expanded database. DeepSleepNet-Lite achieves slightly lower performance, if not on par, compared to the existing state-of-the-art architectures, in overall accuracy, macro F1-score and Cohen's kappa (on Sleep-EDF v1-2013 +/-30mins: 84.0%, 78.0%, 0.78; on Sleep-EDF v2-2018 +/-30mins: 80.3%, 75.2%, 0.73). Monte Carlo dropout enables the estimate of the uncertain predictions. By rejecting the uncertain instances, the model achieves higher performance on both versions of the database (on Sleep-EDF v1-2013 +/-30mins: 86.1.0%, 79.6%, 0.81; on Sleep-EDF v2-2018 +/-30mins: 82.3%, 76.7%, 0.76). Our lighter sleep scoring approach paves the way to the application of scoring algorithms for sleep analysis in real-time.

Motivation & Objective

• To develop a simplified, lightweight deep learning model for automatic sleep staging that reduces computational demands.
• To enable real-time and home-monitoring applications by processing only 90-second EEG sequences.
• To estimate model uncertainty during inference without additional architecture or training overhead.
• To improve model robustness and performance by rejecting uncertain predictions using uncertainty estimates.

Proposed method

• Employs a simplified feed-forward CNN with two parallel branches using small and large filters to capture temporal and frequency patterns.
• Processes 90-second single-channel EEG (Fpz-Cz) sequences, reducing input length compared to prior models.
• Applies Monte Carlo dropout at test time to estimate predictive uncertainty by sampling multiple forward passes.
• Uses label smoothing with a conditional probability distribution derived from prior knowledge to improve model calibration.
• Implements a query procedure to identify and reject uncertain predictions, improving performance on remaining samples.
• Trains the model with batch normalization and ReLU activations, minimizing parameters to reduce overfitting on small datasets.

Experimental results

Research questions

• RQ1 Can a lightweight, 90-second input architecture achieve performance comparable to state-of-the-art models in sleep staging?
• RQ2 Can Monte Carlo dropout provide reliable uncertainty estimates without architectural modifications?
• RQ3 Does rejecting uncertain predictions improve overall accuracy and F1-score in sleep staging?
• RQ4 How does label smoothing with prior knowledge affect model calibration and performance?
• RQ5 Can a minimal architecture with low parameter count generalize well on small datasets without overfitting?

Key findings

• DeepSleepNet-Lite achieves 84.0% accuracy, 78.0% macro F1, and 0.78 Cohen’s kappa on Sleep-EDF v1-2013 (±30mins), matching state-of-the-art performance.
• After rejecting uncertain predictions using Monte Carlo dropout, performance improves to 86.1% accuracy, 79.6% macro F1, and 0.81 kappa on Sleep-EDF v1-2013.
• On Sleep-EDF v2-2018, the model reaches 80.3% accuracy and 0.73 kappa overall, improving to 82.3% accuracy and 0.76 kappa after uncertainty rejection.
• The uncertainty estimation procedure successfully identifies the most challenging sleep stages, particularly N1 and N3, which are often misclassified.
• Label smoothing with prior knowledge improves calibration but does not consistently enhance performance or uncertainty estimation when combined with Monte Carlo dropout.
• The model’s low parameter count (~0.6M) makes it robust to overfitting and suitable for real-time and home-monitoring applications.