[Paper Review] High Dimensional Semiparametric Latent Graphical Model for Mixed Data

Jianqing Fan, Han Liu+2 more|arXiv (Cornell University)|Apr 29, 2014

Statistical Methods and Inference50 references4 citations

TL;DR

This paper proposes a high-dimensional semiparametric latent Gaussian copula model for mixed data, where binary and discrete variables are assumed to arise from latent continuous variables with Gaussian copula dependence. Using rank-based estimation, the method achieves optimal convergence rates for precision matrix and eigenvector estimation as if latent variables were observed, enabling consistent graph recovery and sparse principal component analysis under high-dimensional settings.

ABSTRACT

Graphical models are commonly used tools for modeling multivariate random variables. While there exist many convenient multivariate distributions such as Gaussian distribution for continuous data, mixed data with the presence of discrete variables or a combination of both continuous and discrete variables poses new challenges in statistical modeling. In this paper, we propose a semiparametric model named latent Gaussian copula model for binary and mixed data. The observed binary data are assumed to be obtained by dichotomizing a latent variable satisfying the Gaussian copula distribution or the nonparanormal distribution. The latent Gaussian model with the assumption that the latent variables are multivariate Gaussian is a special case of the proposed model. A novel rank-based approach is proposed for both latent graph estimation and latent principal component analysis. Theoretically, the proposed methods achieve the same rates of convergence for both precision matrix estimation and eigenvector estimation, as if the latent variables were observed. Under similar conditions, the consistency of graph structure recovery and feature selection for leading eigenvectors is established. The performance of the proposed methods is numerically assessed through simulation studies, and the usage of our methods is illustrated by a genetic dataset.

Motivation & Objective

• To address the challenge of modeling mixed data (continuous and discrete variables) in high-dimensional graphical models.
• To develop a semiparametric approach that models discrete variables as censored versions of latent continuous variables with Gaussian copula dependence.
• To enable consistent estimation of conditional independence structures among latent variables, offering deeper insight than observed variable dependencies.
• To establish theoretical guarantees for latent graph estimation and sparse principal component analysis under high-dimensional, low-sample-size settings.
• To provide a rank-based inference method that achieves the same convergence rates as if latent variables were directly observed.

Proposed method

• Proposes a latent Gaussian copula model where observed discrete variables are generated by thresholding latent continuous variables with a Gaussian copula structure.
• Uses rank-based statistics to estimate the latent correlation matrix and precision matrix without assuming parametric forms for marginal distributions.
• Applies a rank-based graphical lasso for latent graph estimation, leveraging Kendall’s tau to approximate latent correlations.
• Develops a rank-based sparse PCA method to estimate leading eigenvectors of the latent covariance matrix under sparsity assumptions.
• Theoretical analysis shows that the convergence rates of precision matrix and eigenvector estimators match those achievable with fully observed latent variables.
• Employs concentration inequalities and matrix perturbation theory to establish consistency in graph structure recovery and feature selection.

Experimental results

Research questions

• RQ1 Can a semiparametric latent variable model effectively handle mixed data types (continuous and discrete) in high-dimensional settings?
• RQ2 Does rank-based estimation of latent correlations achieve the same convergence rates as parametric methods when latent variables are observed?
• RQ3 Can the proposed method consistently recover the true conditional independence structure (graph) among latent variables?
• RQ4 To what extent does the rank-based sparse PCA method recover the true leading eigenvectors under sparsity assumptions?
• RQ5 How robust is the method to deviations from the Gaussian copula assumption, particularly in discrete or count data?

Key findings

• The proposed rank-based method achieves the same convergence rates for precision matrix and eigenvector estimation as if the latent variables were directly observed.
• The method ensures consistent recovery of the latent graph structure and correct feature selection for leading eigenvectors with high probability.
• The estimation error for the latent correlation matrix is bounded by O(√(log d / n)) with high probability, where d is the dimension and n the sample size.
• The method achieves model selection consistency: the probability that the true support of the precision matrix is recovered is greater than 1 - d⁻¹.
• Theoretical results confirm that the rank-based approach is robust to unknown marginal distributions and maintains optimal rates under mild regularity conditions.
• Simulation studies and a real genetic dataset demonstrate the method’s strong finite-sample performance and practical utility in high-dimensional mixed data analysis.