[Paper Review] Deep Explicit Duration Switching Models for Time Series

Abdul Fatir Ansari, Konstantinos Benidis+6 more|arXiv (Cornell University)|Oct 26, 2021

Time Series Analysis and Forecasting46 references4 citations

TL;DR

This paper proposes the Recurrent Explicit Duration Switching Dynamical System (RED-SDS), a deep state space model that jointly captures state-dependent and time-dependent switching dynamics in time series using recurrent state-to-switch connections and explicit duration modeling. It enables efficient hybrid inference via an inference network for continuous states and exact forward-backward inference for discrete switches and counts, achieving state-of-the-art segmentation and competitive forecasting performance on multiple benchmarks.

ABSTRACT

Many complex time series can be effectively subdivided into distinct regimes that exhibit persistent dynamics. Discovering the switching behavior and the statistical patterns in these regimes is important for understanding the underlying dynamical system. We propose the Recurrent Explicit Duration Switching Dynamical System (RED-SDS), a flexible model that is capable of identifying both state- and time-dependent switching dynamics. State-dependent switching is enabled by a recurrent state-to-switch connection and an explicit duration count variable is used to improve the time-dependent switching behavior. We demonstrate how to perform efficient inference using a hybrid algorithm that approximates the posterior of the continuous states via an inference network and performs exact inference for the discrete switches and counts. The model is trained by maximizing a Monte Carlo lower bound of the marginal log-likelihood that can be computed efficiently as a byproduct of the inference routine. Empirical results on multiple datasets demonstrate that RED-SDS achieves considerable improvement in time series segmentation and competitive forecasting performance against the state of the art.

Motivation & Objective

• To address the limitation of existing switching dynamical systems that cannot jointly model state-dependent and time-dependent switching behaviors.
• To improve time series segmentation by capturing long-term temporal patterns and reducing oversegmentation.
• To enable competitive probabilistic forecasting by learning regime-specific dynamics and extrapolating them consistently.
• To develop an efficient inference algorithm that combines approximate inference for continuous states with exact inference for discrete switches and duration counts.
• To incorporate inductive biases via duration model hyperparameters to better identify persistent regimes in complex time series.

Proposed method

• RED-SDS models time series using a latent state that evolves via nonlinear, recurrent transitions, with switches determining the active regime.
• It introduces explicit duration variables to model time-dependent switching, where each switch has a duration distribution (e.g., geometric or gamma) that captures persistence in a regime.
• A recurrent connection from the state to the switch enables state-dependent switching, allowing regime changes based on state trajectory.
• Hybrid inference is used: an inference network approximates the posterior of continuous states, while exact forward-backward algorithms compute posteriors over discrete switches and counts.
• The model is trained by maximizing a Monte Carlo lower bound (ELBO) of the marginal log-likelihood, computed efficiently during the inference routine.
• Neural networks parameterize the transition, emission, and switch dynamics, enabling flexible modeling of nonlinear and non-Gaussian time series.

Experimental results

Research questions

• RQ1 Can a single model effectively capture both state-dependent and time-dependent switching dynamics in time series?
• RQ2 Does explicit duration modeling improve segmentation accuracy and reduce oversegmentation compared to models without it?
• RQ3 Can the hybrid inference framework efficiently handle the increased complexity of joint state- and time-dependent switching?
• RQ4 How does RED-SDS perform in forecasting compared to state-of-the-art models across diverse datasets?
• RQ5 Can the model learn meaningful, interpretable regimes without imposed structural constraints?

Key findings

• RED-SDS significantly outperforms a strong baseline (e.g., [13]) in time series segmentation, reducing oversegmentation and better capturing long-term patterns, as demonstrated on the dancing bees dataset.
• The model achieves competitive forecasting performance, with CRPS results comparable to or better than state-of-the-art models like DeepAR and KVAE on benchmark datasets.
• Explicit duration modeling enables the model to learn persistent regimes and extrapolate them consistently into the forecast horizon, simplifying the forecasting task.
• The hybrid inference algorithm enables efficient training and inference, with the Monte Carlo ELBO computed as a byproduct of the inference routine.
• Hyperparameter tuning of duration model distributions and state dimensionality (2–16) improves performance across datasets, indicating the importance of inductive biases.
• The model demonstrates robustness across diverse time series, including those with complex, mixed switching behaviors such as city power consumption and bee waggle dances.