[Paper Review] Flexible Bayesian Dynamic Modeling of Correlation and Covariance Matrices

Shiwei Lan, Andrew J. Holbrook+4 more|arXiv (Cornell University)|Nov 8, 2017

Bayesian Methods and Mixture Models66 references3 citations

TL;DR

This paper proposes a flexible Bayesian framework for modeling dynamic correlation and covariance matrices using a unit-vector product representation on spheres, enabling non-informative, interpretable priors beyond the restrictive inverse-Wishart. It introduces adaptive Δ-Spherical Hamiltonian Monte Carlo for posterior computation and demonstrates superior detection of neural dynamics in rat memory tasks, revealing previously undetected cognitive state differences before 500ms.

ABSTRACT

Modeling correlation (and covariance) matrices can be challenging due to the positive-definiteness constraint and potential high-dimensionality. Our approach is to decompose the covariance matrix into the correlation and variance matrices and propose a novel Bayesian framework based on modeling the correlations as products of unit vectors. By specifying a wide range of distributions on a sphere (e.g. the squared-Dirichlet distribution), the proposed approach induces flexible prior distributions for covariance matrices (that go beyond the commonly used inverse-Wishart prior). For modeling real-life spatio-temporal processes with complex dependence structures, we extend our method to dynamic cases and introduce unit-vector Gaussian process priors in order to capture the evolution of correlation among components of a multivariate time series. To handle the intractability of the resulting posterior, we introduce the adaptive $Δ$-Spherical Hamiltonian Monte Carlo. We demonstrate the validity and flexibility of our proposed framework in a simulation study of periodic processes and an analysis of rat's local field potential activity in a complex sequence memory task.

Motivation & Objective

• To address the limitations of standard inverse-Wishart priors in covariance matrix modeling, which are restrictive and lack flexibility.
• To develop a computationally efficient Bayesian method that maintains positive-definiteness and interpretability of variance and correlation components.
• To extend the framework to dynamic settings for modeling time-evolving dependence in multivariate stochastic processes, especially in neuroscience applications.
• To introduce a novel inference algorithm, adaptive Δ-Spherical Hamiltonian Monte Carlo, to handle intractable posteriors arising from complex priors.
• To theoretically and empirically investigate posterior contraction rates for dynamic covariance modeling, a first in this domain.

Proposed method

• Decomposes the covariance matrix into variance and correlation matrices, modeling correlations via products of unit vectors on a sphere.
• Uses distributions on the sphere—such as the squared-Dirichlet distribution—to induce flexible, non-conjugate priors on correlation matrices.
• Imposes unit-vector Gaussian process priors on the spherical components to model time-varying correlation structures in dynamic processes.
• Applies Cholesky decomposition to ensure positive-definiteness of the resulting correlation matrix at each time point.
• Develops adaptive Δ-Spherical Hamiltonian Monte Carlo to efficiently sample from the intractable posterior distribution under the novel geometric prior structure.
• Employs a structured parameterization that allows for dependent priors across components while preserving interpretability of pairwise correlations.

Experimental results

Research questions

• RQ1 Can a geometric, unit-vector-based representation on spheres provide a flexible and interpretable alternative to the inverse-Wishart prior for covariance matrices?
• RQ2 How can dynamic correlation structures in multivariate time series be effectively modeled using Gaussian process priors on spherical components?
• RQ3 What is the posterior contraction rate of the proposed Bayesian model for dynamic covariance matrices, and how does it compare to that of the mean function?
• RQ4 Can the proposed method detect subtle, time-varying neural connectivity changes in complex cognitive tasks where traditional methods fail?
• RQ5 How does the adaptive Δ-Spherical HMC algorithm perform in terms of mixing and convergence for high-dimensional, constrained posterior distributions?

Key findings

• The proposed method successfully detected significant differences in neural activity between InSeq and OutSeq trials in a rat memory task, particularly before 500ms, which were undetected by conventional analyses.
• The model revealed distinct neural dynamics in the time window before 350ms, indicating early processing of sequence information, even before behavioral responses.
• Posterior contraction analysis showed that the covariance posterior contracts at a rate consistent with concentration functions, though slower than the mean posterior, suggesting room for theoretical refinement.
• The adaptive Δ-Spherical HMC algorithm enabled efficient sampling from the complex, high-dimensional posterior, supporting practical application to real neural data.
• The framework outperformed standard latent factor and parametric models in capturing complex, non-stationary dependence in simulated periodic processes and real LFP data.
• The method demonstrated robustness and sensitivity in detecting dynamic brain connectivity changes, especially in early cognitive processing stages, highlighting its utility in neuroscience.