[Paper Review] Nonparametric Compressive Graphical Model Selection for Vector-Valued Stationary Random Processes: A Multitask Learning Approach

Alexander Jung|arXiv (Cornell University)|Apr 4, 2014

Control Systems and Identification36 references3 citations

TL;DR

This paper proposes a nonparametric, compressive method for learning the conditional independence graph (CIG) of high-dimensional vector-valued stationary Gaussian time series without assuming a parametric model like AR. It leverages smoothness in the Fourier domain and guarantees asymptotic consistency with high probability, outperforming existing methods under model mismatch.

ABSTRACT

We propose a method for inferring the conditional independence graph (CIG) of a high-dimensional Gaussian time series (discrete time process) from a finite-length observation. By contrast to existing approaches, we do not rely on a parametric process model (such as, e.g., an autoregressive model) for the observed random process. Instead, we only require certain smoothness properties (in the Fourier domain) of the process only. The proposed inference scheme is compressive in that it works even for sample sizes much smaller than the number of scalar process components. A theoretical performance analysis provides conditions which guarantee that the probability of the proposed inference method to deliver a wrong the CIG is below a prescribed value. This analysis reveals conditions for the new method to be consistent asymptotically. Some numerical experiments validate our theoretical performance analysis and demonstrate superior performance of our scheme compared to existing approaches in case of model mismatch.

Motivation & Objective

• To infer the conditional independence graph (CIG) of high-dimensional Gaussian time series from short observations without assuming a parametric model.
• To develop a compressive inference scheme that works when sample size is much smaller than the number of process components.
• To establish theoretical conditions ensuring the probability of incorrect CIG recovery is below a prescribed threshold.
• To achieve asymptotic consistency of the inference method under minimal assumptions on the underlying process.
• To validate performance superiority over existing methods in scenarios involving model mismatch.

Proposed method

• The method operates in the Fourier domain, exploiting smoothness properties of the spectral density matrix to avoid parametric assumptions like autoregressive models.
• It formulates the CIG learning problem as a multitask learning task, enabling joint estimation across frequency components.
• The inference scheme is compressive, relying on low-rank and sparse structure in the precision matrix to enable recovery from limited samples.
• A theoretical analysis bounds the probability of error in CIG recovery using concentration inequalities under smoothness constraints.
• The method uses a nonparametric estimator of the spectral density matrix that respects the smoothness of the process in the frequency domain.
• The final CIG is recovered via thresholding of the estimated precision matrix, with tuning based on the theoretical error bounds.

Experimental results

Research questions

• RQ1 Can a nonparametric method infer the conditional independence graph of a high-dimensional stationary Gaussian process without assuming a parametric model like AR?
• RQ2 Under what conditions does the proposed compressive inference method achieve asymptotic consistency in CIG recovery?
• RQ3 How does the method perform in finite-sample regimes when the true process model is misspecified compared to parametric alternatives?
• RQ4 What theoretical guarantees can be provided on the probability of error in CIG estimation under smoothness assumptions in the Fourier domain?
• RQ5 Can multitask learning principles be effectively combined with compressive sensing to improve CIG estimation in high-dimensional time series?

Key findings

• The proposed method achieves asymptotic consistency in CIG recovery under mild smoothness assumptions on the spectral density matrix in the Fourier domain.
• Theoretical analysis provides explicit conditions under which the probability of incorrect CIG recovery is bounded below a user-prescribed threshold.
• Numerical experiments demonstrate superior performance compared to parametric methods when the true process model is misspecified.
• The method remains effective even when the number of samples is much smaller than the dimension of the process components, confirming its compressive nature.
• The use of multitask learning across frequency components enhances estimation accuracy and robustness in high-dimensional settings.
• The method does not require knowledge of the true parametric model (e.g., AR order), making it more robust in practice than model-based alternatives.