[Paper Review] Dynamic generalized linear models for non-Gaussian time series forecasting

Kostas Triantafyllopoulos|ArXiv.org|Feb 1, 2008

Statistical Methods and Bayesian Inference42 references3 citations

TL;DR

This paper presents a unified Bayesian framework for forecasting non-Gaussian time series using dynamic generalized linear models (DGLMs), employing conjugate priors and Bayes linear estimation to enable multi-step forecasting across diverse exponential family distributions. The key contribution is a systematic, computationally efficient approach to forecasting and model monitoring applicable across finance, medicine, biology, and behavioral sciences.

ABSTRACT

The purpose of this paper is to provide a discussion, with illustrating examples, on Bayesian forecasting for dynamic generalized linear models (DGLMs). Adopting approximate Bayesian analysis, based on conjugate forms and on Bayes linear estimation, we describe the theoretical framework and then we provide detailed examples of response distributions, including binomial, Poisson, negative binomial, geometric, normal, log-normal, gamma, exponential, Weibull, Pareto, beta, and inverse Gaussian. We give numerical illustrations for all distributions (except for the normal). Putting together all the above distributions, we give a unified Bayesian approach to non-Gaussian time series analysis, with applications from finance and medicine to biology and the behavioural sciences. Throughout the models we discuss Bayesian forecasting and, for each model, we derive the multi-step forecast mean. Finally, we describe model assessment using the likelihood function, and Bayesian model monitoring.

Motivation & Objective

• To develop a comprehensive Bayesian forecasting framework for non-Gaussian time series using dynamic generalized linear models (DGLMs).
• To extend existing DGLM approaches by incorporating approximate Bayesian inference, multi-step forecasting, and model diagnostics beyond the commonly used binomial and Poisson models.
• To unify forecasting across a wide range of exponential family distributions, including binomial, Poisson, negative binomial, gamma, inverse Gaussian, and beta, among others.
• To provide numerical illustrations using real and simulated data for all distributions except the normal, which is already well-documented.
• To introduce likelihood-based model assessment and Bayesian model monitoring for DGLMs, enhancing model reliability and interpretability.

Proposed method

• Adopting conjugate priors and Bayes linear estimation for approximate Bayesian inference in DGLMs, enabling tractable posterior updates.
• Using the state space formulation with time-varying parameters: $\theta_t = G_t \theta_{t-1} + \omega_t$, where $\omega_t \sim N(0, \Omega_t)$.
• Applying the Kalman filter framework to update the posterior distribution of the linear predictor $\eta_t = F_t' \theta_t$ sequentially over time.
• Deriving multi-step forecast means via the posterior predictive distribution, using the posterior mean and variance of $\eta_t$.
• Employing the cumulant generating function and its derivatives to approximate posterior moments of the natural parameter $\gamma_t$, especially for non-conjugate or complex distributions.
• Implementing model monitoring through the likelihood function and posterior predictive checks, with specific attention to predictive density evaluation and residual analysis.

Experimental results

Research questions

• RQ1 How can a unified Bayesian forecasting framework be developed for non-Gaussian time series across diverse exponential family distributions?
• RQ2 What are the multi-step forecast mean and variance expressions for DGLMs under conjugate priors and approximate Bayesian updating?
• RQ3 How can model assessment and monitoring be systematically performed in DGLMs using likelihood-based and Bayesian diagnostics?
• RQ4 What are the computational and statistical challenges in extending DGLMs beyond binomial and Poisson models, and how can they be addressed?
• RQ5 How do the approximations for the digamma and trigamma functions improve the accuracy of posterior mean and variance estimates in DGLMs?

Key findings

• The paper derives explicit expressions for the multi-step forecast mean in DGLMs using the posterior mean of the linear predictor $\eta_t$, enabling reliable long-term forecasting.
• For the beta distribution, the posterior mean of $\mu_t$ is derived as $\mathbb{E}(\mu_t|y^{t-1}) = \frac{r_t}{r_t + s_t}$, where $r_t$ and $s_t$ are updated via the prior parameters.
• For the inverse Gaussian distribution, the posterior mean of $\mu_t$ is obtained through transformation of the natural parameter $\gamma_t = -1/\mu_t^2$, with the posterior expectation computed via integration of the non-standard density.
• The use of asymptotic approximations for the digamma function $\psi(x) \approx \log x$ and its derivative $\psi^{(1)}(x) \approx 1/x$ enables stable computation of posterior moments even for small parameter values.
• The posterior variance of $\eta_t$ is approximated as $q_t = \text{Var}(\eta_t|y^{t-1}) = \frac{1}{r_t} - \frac{1}{s_t - r_t}$ for the beta distribution, with similar expressions derived for other distributions.
• Model monitoring is facilitated through the likelihood function and posterior predictive checks, with numerical illustrations showing consistent performance across all 12 distributions tested (excluding normal).