[Paper Review] dpMood: Exploiting Local and Periodic Typing Dynamics for Personalized Mood Prediction

He Huang, Bokai Cao+3 more|arXiv (Cornell University)|Aug 29, 2018

Mental Health Research Topics28 references3 citations

TL;DR

This paper proposes dpMood, a deep learning model that leverages local typing dynamics and circadian rhythms from smartphone meta-data to enable personalized mood prediction for bipolar disorder. By fusing accelerometer and keystroke data through early fusion of CNNs and RNNs, and calibrating predictions per individual using time-based modeling, dpMood achieves state-of-the-art performance in predicting HDRS and YMRS scores with lower regression error than prior methods.

ABSTRACT

Mood disorders are common and associated with significant morbidity and mortality. Early diagnosis has the potential to greatly alleviate the burden of mental illness and the ever increasing costs to families and society. Mobile devices provide us a promising opportunity to detect the users' mood in an unobtrusive manner. In this study, we use a custom keyboard which collects keystrokes' meta-data and accelerometer values. Based on the collected time series data in multiple modalities, we propose a deep personalized mood prediction approach, called {\pro}, by integrating convolutional and recurrent deep architectures as well as exploring each individual's circadian rhythm. Experimental results not only demonstrate the feasibility and effectiveness of using smart-phone meta-data to predict the presence and severity of mood disturbances in bipolar subjects, but also show the potential of personalized medical treatment for mood disorders.

Motivation & Objective

• To develop a personalized mood prediction system using unobtrusive smartphone sensor data for individuals with bipolar disorder.
• To address the challenge of capturing local patterns in typing dynamics and circadian rhythms in mood states.
• To improve prediction accuracy by integrating multi-modal smartphone data (keystrokes and accelerometer) through deep learning.
• To explore early versus late fusion strategies for multi-view time-series data in mood prediction.
• To calibrate predictions per individual using temporal patterns to reflect personalized circadian rhythms.

Proposed method

• The model employs a late fusion architecture combining convolutional neural networks (CNNs) to capture local patterns in typing sequences and recurrent neural networks (RNNs) to model long-term temporal dynamics.
• Early fusion of keystroke and accelerometer features is explored and shown to outperform late fusion in mood prediction performance.
• A time-based calibration mechanism is applied per user to model individual circadian rhythms, improving personalization and prediction accuracy.
• The model uses a two-stream encoder: one for keystroke meta-data (e.g., keypress duration, interval) and one for accelerometer data, followed by feature concatenation.
• The final prediction head performs regression on Hamilton Depression Rating Scale (HDRS) and Young Mania Rating Scale (YMRS) scores.
• The architecture is trained end-to-end using multi-task learning to predict both depression and mania severity simultaneously.

Experimental results

Research questions

• RQ1 Can local patterns in smartphone typing dynamics predict mood states in individuals with bipolar disorder?
• RQ2 How does incorporating an individual’s circadian rhythm improve personalized mood prediction accuracy?
• RQ3 Does early fusion of multi-modal smartphone data (keystrokes and accelerometer) outperform late fusion in mood prediction?
• RQ4 To what extent can deep learning models combining CNNs and RNNs capture both local and global temporal patterns in typing dynamics for mood prediction?
• RQ5 Can time-based calibration per user enhance the precision of mood severity estimation?

Key findings

• dpMood achieves the lowest regression error on both HDRS and YMRS prediction tasks compared to baseline models, demonstrating superior performance in mood severity estimation.
• Early fusion of keystroke and accelerometer features significantly improves prediction accuracy over late fusion, indicating that joint feature learning is more effective.
• Each subject exhibits a unique circadian rhythm, with distinct sine wave patterns in their mood dynamics, confirming individual variability in biological timing.
• The offset term δu in the circadian model is clustered by diagnosis: controls have positive offsets, while bipolar I and II subjects show negative offsets, aligning with clinical expectations.
• The model successfully captures individual-specific patterns in typing dynamics, enabling precise, personalized mood prediction without accessing text content.
• The open-source code for dpMood is publicly available, supporting reproducibility and further research in digital phenotyping.