[Paper Review] Bayesian Dynamic Fused LASSO

Kaoru Irie|arXiv (Cornell University)|May 29, 2019

Statistical Methods and Inference47 references4 citations

TL;DR

This paper introduces the Bayesian Dynamic Fused LASSO (DFL), a novel Markov transition prior that jointly shrinks time-varying regression coefficients toward zero and their previous values, enabling dynamic sparsity in state-space models. By deriving the intractable normalizing constant as a log-geometric mixture of double-exponentials, the DFL enables efficient posterior computation via forward filtering and backward sampling in conditionally Gaussian dynamic linear models, outperforming standard priors in shrinkage flexibility and predictive accuracy.

ABSTRACT

The new class of Markov processes is proposed to realize the flexible shrinkage effects for the dynamic models. The transition density of the new process consists of two penalty functions, similarly to Bayesian fused LASSO in its functional form, that shrink the current state variable to its previous value and zero. The normalizing constant of the density, which is not ignorable in the posterior computation, is shown to be essentially the log-geometric mixture of double-exponential densities. This process comprises the state equation of the dynamic regression models, which is shown to be conditionally Gaussian and linear in state variables and utilize the forward filtering and backward sampling in posterior computation by Gibbs sampler. The problem of overshrinkage that is inherent in lasso is moderated by considering the hierarchical extension, which can even realize the shrinkage of horseshoe priors marginally. The new prior is compared with the standard double-exponential prior in the estimation of and prediction by the dynamic linear models for illustration. It is also applied to the time-varying vector autoregressive models for the US macroeconomic data, where we examine the (dis)similarity of the additional shrinkage effect to dynamic variable selection or, specifically, the latent threshold models.

Motivation & Objective

• To develop a prior that enables flexible, time-varying shrinkage of dynamic coefficients toward both zero and their previous values, addressing overshrinkage in standard LASSO.
• To explicitly compute the intractable normalizing constant of a conditional transition density with two penalty functions, which is critical for posterior inference.
• To embed the DFL prior into a conditionally Gaussian dynamic linear model (CDLM) framework to enable efficient posterior sampling using forward filtering and backward sampling (FFBS).
• To moderate the overshrinkage inherent in lasso-type priors by introducing a hierarchical extension that can mimic horseshoe-like marginal shrinkage.
• To demonstrate the DFL’s effectiveness in dynamic linear models and time-varying vector autoregressions (TV-VARs) for US macroeconomic data.

Proposed method

• Proposes a transition density for state variables: $ p(x_t|x_{t-1}) \propto \exp\{ -\alpha|x_t| - \beta|x_t - x_{t-1}| \} $, combining shrinkage toward zero and the previous state.
• Derives the normalizing constant as a log-geometric mixture of double-exponential densities, enabling analytical tractability.
• Establishes a hierarchical representation of the DFL prior as a scale mixture of normals, facilitating MCMC via Gibbs sampling.
• Embeds the DFL into a conditionally Gaussian dynamic linear model (CDLM), ensuring conjugacy and enabling efficient forward filtering and backward sampling (FFBS).
• Utilizes a hierarchical extension of the DFL prior to allow for marginally horseshoe-like shrinkage, reducing overshrinkage.
• Applies the method to univariate DLMs and multivariate TV-VARs, comparing performance with standard double-exponential and latent threshold models.

Experimental results

Research questions

• RQ1 Can a conditional prior that shrinks coefficients toward both zero and their previous values be constructed with a tractable normalizing constant?
• RQ2 How can the DFL prior be embedded into a CDLM to enable efficient posterior computation via FFBS?
• RQ3 Does the DFL prior reduce overshrinkage compared to standard LASSO while maintaining sparsity in dynamic models?
• RQ4 How does the DFL perform in estimating time-varying parameters in multivariate macroeconomic models compared to existing shrinkage priors?
• RQ5 To what extent does the DFL prior capture dynamic sparsity and structural changes in time-varying vector autoregressions?

Key findings

• The normalizing constant of the DFL transition density is shown to be a log-geometric mixture of double-exponential densities, enabling analytical computation.
• The DFL prior is embedded into a CDLM framework, allowing efficient posterior sampling via forward filtering and backward sampling (FFBS).
• In simulation studies, the DFL-DLM shows improved estimation and prediction accuracy compared to standard double-exponential priors, particularly in dynamic sparsity detection.
• In US macroeconomic data, the DFL-DLM identifies sparse, time-varying structures with posterior probabilities of inclusion close to zero for many coefficients, indicating strong shrinkage toward zero.
• The DFL-DLM exhibits more dynamic shrinkage than the latent threshold model (LTM), with posterior probabilities of positive counts being less static and more responsive to structural changes.
• The DFL prior allows for marginally horseshoe-like shrinkage through hierarchical extension, moderating the overshrinkage typical of standard LASSO.