[Paper Review] Diffusion Models for Time Series Applications: A Survey

Lequan Lin, Zhengkun Li+3 more|arXiv (Cornell University)|May 1, 2023

Complex Systems and Time Series AnalysisEconomics, Econometrics and Finance9 citations

TL;DR

A survey summarizing diffusion-based models for time series forecasting, imputation, and generation, including DDPMs, SGMs, and SDEs, with connections to TimeGrad, ScoreGrad, and related methods.

ABSTRACT

Diffusion models, a family of generative models based on deep learning, have become increasingly prominent in cutting-edge machine learning research. With a distinguished performance in generating samples that resemble the observed data, diffusion models are widely used in image, video, and text synthesis nowadays. In recent years, the concept of diffusion has been extended to time series applications, and many powerful models have been developed. Considering the deficiency of a methodical summary and discourse on these models, we provide this survey as an elementary resource for new researchers in this area and also an inspiration to motivate future research. For better understanding, we include an introduction about the basics of diffusion models. Except for this, we primarily focus on diffusion-based methods for time series forecasting, imputation, and generation, and present them respectively in three individual sections. We also compare different methods for the same application and highlight their connections if applicable. Lastly, we conclude the common limitation of diffusion-based methods and highlight potential future research directions.

Motivation & Objective

• Survey diffusion-model-based methods applied to time series to provide a systematic overview for new researchers.
• Explain core diffusion formulations (DDPMs, SGMs, SDEs) and how they are adapted for time series tasks.
• Categorize methods by application: forecasting, imputation, generation, and discuss spatio-temporal extensions.

Proposed method

• Explain the forward and reverse diffusion processes and how neural networks model the reverse transition.
• Present DDPM, SGMs, and SDE perspectives with key equations and simplifications (e.g., Gaussian noise, score matching).
• Describe how TimeGrad and ScoreGrad adapt diffusion for multivariate time series forecasting via conditioning on history.
• Discuss D3VAE’s coupled diffusion and BVAE framework for noisy/limited data and latent disentanglement.
• Introduce DSPD and CSPD as continuous-process diffusion variants for time series with continuous noise processes.

Experimental results

Research questions

• RQ1 What are the main diffusion-model formulations applicable to time series (DDPMs, SGMs, SDEs) and how do they differ conceptually and computationally?
• RQ2 How are diffusion models adapted for time series forecasting, imputation, and generation, including conditioning on history and covariates?
• RQ3 What are the connections and distinctions among prominent models (TimeGrad, ScoreGrad, D3VAE, DSPD, CSPD) in their problem formulations and objectives?
• RQ4 What limitations and future directions arise when applying diffusion models to time series tasks?

Key findings

• Diffusion-model frameworks can be tailored to time series via forward noise injection and learned reverse denoising, enabling forecasting, imputation, and generation.
• DDPM, SGM, and SDE formulations are unified under time series contexts, with variants like VP/VE SDEs and predictor-corrector samplers guiding sampling.
• TimeGrad and ScoreGrad illustrate conditioning on historical context and covariates to capture temporal dynamics in multivariate series.
• D3VAE introduces coupled diffusion for data augmentation and latent disentanglement to improve robustness with limited/noisy data.
• DSPD generalizes DDPM to continuous-time processes using Gaussian processes as the noise function, enabling continuous-time forecasting and efficient multi-point sampling.
• CSPD extends CSPD to continuous stochastic processes, integrating continuous diffusion with time series diffusion.]
• The survey highlights common limitations and outlines potential future research directions in diffusion-based time series methods.