[Paper Review] DynaConF: Dynamic Forecasting of Non-Stationary Time Series

Siqi Liu, Andreas Lehrmann|arXiv (Cornell University)|Sep 17, 2022

Time Series Analysis and Forecasting4 citations

TL;DR

DynaConF proposes a novel deep learning framework for forecasting non-stationary time series by explicitly decoupling stationary conditional distribution modeling from time-variant non-stationary dynamics using a Bayesian dynamic model and a factorized deep conditional distribution network. It outperforms state-of-the-art models in both synthetic and real-world datasets, especially under dynamic distribution shifts, achieving lower CRPS and MSE scores by effectively adapting to unseen changes.

ABSTRACT

Deep learning has shown impressive results in a variety of time series forecasting tasks, where modeling the conditional distribution of the future given the past is the essence. However, when this conditional distribution is non-stationary, it poses challenges for these models to learn consistently and to predict accurately. In this work, we propose a new method to model non-stationary conditional distributions over time by clearly decoupling stationary conditional distribution modeling from non-stationary dynamics modeling. Our method is based on a Bayesian dynamic model that can adapt to conditional distribution changes and a deep conditional distribution model that handles multivariate time series using a factorized output space. Our experimental results on synthetic and real-world datasets show that our model can adapt to non-stationary time series better than state-of-the-art deep learning solutions.

Motivation & Objective

• To address the challenge of forecasting time series with non-stationary conditional distributions, which current deep learning models struggle with due to implicit stationarity assumptions.
• To develop a method that explicitly models changes in the conditional distribution over time, enabling better generalization to unseen distribution shifts.
• To decouple time-invariant (stationary) distribution modeling from time-variant (non-stationary) dynamics modeling for improved adaptability and inference efficiency.
• To enable robust test-time adaptation to new, previously unseen distributional changes in the conditional distribution of time series.
• To improve predictive performance on multivariate time series with evolving statistical properties, especially when underlying dynamics shift due to unobserved or novel events.

Proposed method

• The model uses a Bayesian dynamic model to represent time-variant changes in the conditional distribution, allowing for adaptive learning of non-stationary dynamics.
• It employs a deep conditional distribution model with a factorized output space to efficiently model multivariate time series and learn a stationary conditional distribution given a control variable.
• A variational posterior is used to approximate the posterior over latent states, with complexity scaling linearly in time, enabling efficient training on long sequences.
• The framework integrates Rao-Blackwellized particle filters to enable efficient inference and posterior estimation under non-stationary conditions.
• The control variable explicitly parameterizes the time-variant component, allowing the model to disentangle stationary distribution learning from dynamic shift modeling.
• The model is trained end-to-end using a differentiable variational inference objective that balances reconstruction and posterior regularization.

Experimental results

Research questions

• RQ1 Can a deep learning model effectively learn and adapt to non-stationary conditional distributions in time series without assuming time-invariant parameters?
• RQ2 How does decoupling stationary distribution modeling from non-stationary dynamics modeling improve generalization to unseen distribution shifts?
• RQ3 To what extent can the proposed method outperform state-of-the-art models on time series with abrupt or continuous distributional changes?
• RQ4 How does the model perform on real-world datasets where non-stationarity arises from unobserved or novel events?
• RQ5 What is the impact of explicit modeling of time-variant dynamics on predictive uncertainty and robustness in test-time adaptation?

Key findings

• On synthetic data, DynaConF achieved a test CRPS of 0.301 and MSE of 3.7, outperforming all baselines, including the ablated StatiConF model, which had higher error, indicating the necessity of dynamic modeling.
• On the Walmart dataset, DynaConF achieved a CRPS of 0.512 and MSE of 2.497e-3, significantly outperforming the best baseline (TimeGrad) with CRPS 0.604 and MSE 3.190e-2.
• On the Temperature dataset, DynaConF achieved a CRPS of 0.186 and MSE of 1.017e-3, outperforming TimeGrad (CRPS 0.195, MSE 1.077e-3) and other baselines.
• The model demonstrated consistent performance across diverse datasets, with the largest gains observed in settings with significant conditional distribution shifts, confirming its design goal of handling non-stationarity.
• Abllation studies showed that StatiConF (the stationary-only variant) underperformed DynaConF, proving that explicit modeling of time-variant dynamics is essential for non-stationary settings.
• The model successfully adapted to unseen distribution shifts at test time, as evidenced by lower predictive uncertainty and improved calibration metrics, particularly in dynamic environments.