[Paper Review] T-LoHo: A Bayesian Regularization Model for Structured Sparsity and Smoothness on Graphs

Changwoo J. Lee, Zhao Tang Luo+1 more|arXiv (Cornell University)|Jul 6, 2021

Statistical Methods and Inference45 references4 citations

TL;DR

T-LoHo proposes a Bayesian hierarchical model using a tree-based low-rank horseshoe prior to induce structured sparsity and smoothness in graph-structured high-dimensional parameters. By leveraging a random spanning forest to define local shrinkage structures, it enables efficient MCMC inference with uncertainty quantification, outperforming fused lasso and other penalized methods in simulation and real-world anomaly detection on road networks.

ABSTRACT

Graphs have been commonly used to represent complex data structures. In models dealing with graph-structured data, multivariate parameters may not only exhibit sparse patterns but have structured sparsity and smoothness in the sense that both zero and non-zero parameters tend to cluster together. We propose a new prior for high-dimensional parameters with graphical relations, referred to as the Tree-based Low-rank Horseshoe (T-LoHo) model, that generalizes the popular univariate Bayesian horseshoe shrinkage prior to the multivariate setting to detect structured sparsity and smoothness simultaneously. The T-LoHo prior can be embedded in many high-dimensional hierarchical models. To illustrate its utility, we apply it to regularize a Bayesian high-dimensional regression problem where the regression coefficients are linked by a graph, so that the resulting clusters have flexible shapes and satisfy the cluster contiguity constraint with respect to the graph. We design an efficient Markov chain Monte Carlo algorithm that delivers full Bayesian inference with uncertainty measures for model parameters such as the number of clusters. We offer theoretical investigations of the clustering effects and posterior concentration results. Finally, we illustrate the performance of the model with simulation studies and a real data application for anomaly detection on a road network. The results indicate substantial improvements over other competing methods such as the sparse fused lasso.

Motivation & Objective

• To develop a Bayesian regularization model that captures both structured sparsity and smoothness in high-dimensional parameters linked by a graph.
• To overcome computational limitations of graph-based regularization in large graphs by using a spanning forest to simplify dependency structure.
• To provide full Bayesian inference with uncertainty measures for cluster count and parameter estimates, unlike frequentist penalized methods.
• To enable adaptive clustering with flexible shapes that respect graph contiguity, avoiding over-clustering from fixed chain orders.
• To demonstrate superior performance in detecting anomalies in spatial and network data, such as traffic changes during events.

Proposed method

• Introduces the Tree-based Low-rank Horseshoe (T-LoHo) prior, a multivariate extension of the univariate horseshoe prior that induces piecewise-constant behavior on a graph.
• Uses a random spanning forest (RSF) to define a compatible neighbor ordering, reducing computational complexity while preserving graph structure.
• Implements a hierarchical prior where local shrinkage parameters are low-rank and structured, enabling joint clustering and sparsity.
• Employs an efficient MCMC algorithm with Gibbs sampling and slice sampling to generate posterior samples and quantify uncertainty.
• Applies the prior to high-dimensional linear regression with graph-structured coefficients, allowing for flexible, contiguity-respecting clusters.
• Incorporates a global-local shrinkage mechanism with a hierarchical structure to adaptively shrink irrelevant coefficients while preserving true signals.

Experimental results

Research questions

• RQ1 Can a Bayesian prior be designed to jointly induce structured sparsity and smoothness in graph-structured high-dimensional parameters?
• RQ2 How can computational efficiency be improved in graph-regularized Bayesian models without sacrificing model flexibility?
• RQ3 Can the proposed method outperform frequentist penalized methods like fused lasso in detecting clustered signals with uncertainty quantification?
• RQ4 How does the use of a random spanning forest compare to fixed chain or tree orders in preserving true cluster structure?
• RQ5 To what extent does the model adapt to complex, non-convex cluster shapes in real-world network data?

Key findings

• T-LoHo outperforms sparse fused lasso and other competing methods in both predictive accuracy and clustering accuracy across all (ϑ, SNR) settings in simulation studies.
• In the NYC Pride March anomaly detection, T-LoHo successfully captured reduced taxi activity along the parade route and in lower Manhattan, while fused lasso failed due to bias from soft-thresholding.
• T-LoHo produced reliable 90% credible intervals, demonstrating its ability to quantify uncertainty, unlike optimization-based methods.
• The model detected subtle spatial patterns around event start/end points, showing higher sensitivity than FL.
• The use of a random spanning forest enabled adaptive clustering with flexible shapes, avoiding over-clustering common in fixed-order methods.
• Theoretical results confirm posterior concentration and clustering effects, supporting the model’s consistency and robustness.