[Paper Review] Sparse Estimation of Multivariate Poisson Log-Normal Models from Count Data

Hao Wu, Xinwei Deng+1 more|arXiv (Cornell University)|Feb 22, 2016

Statistical Methods and Bayesian Inference30 references3 citations

TL;DR

This paper proposes a sparse multivariate Poisson log-normal regression model for multivariate count data, using a Monte Carlo EM algorithm to jointly estimate regression coefficients and the inverse covariance matrix of latent variables. The method improves prediction by leveraging conditional dependencies among responses through Lasso and graphical Lasso penalties, demonstrating superior performance on synthetic and real influenza-like illness data.

ABSTRACT

Modeling data with multivariate count responses is a challenging problem due to the discrete nature of the responses. Existing methods for univariate count responses cannot be easily extended to the multivariate case since the dependency among multiple responses needs to be properly accommodated. In this paper, we propose a multivariate Poisson log-normal regression model for multivariate data with count responses. By simultaneously estimating the regression coefficients and inverse covariance matrix over the latent variables with an efficient Monte Carlo EM algorithm, the proposed regression model takes advantages of association among multiple count responses to improve the model prediction performance. Simulation studies and applications to real world data are conducted to systematically evaluate the performance of the proposed method in comparison with conventional methods.

Motivation & Objective

• To address the challenge of modeling multivariate count responses with complex dependencies that univariate or continuous-response models cannot capture.
• To develop a joint estimation framework that simultaneously learns regression coefficients and the inverse covariance matrix of latent variables to improve prediction accuracy.
• To incorporate sparsity in both regression coefficients and the precision matrix using regularization, enabling feature selection and dependency structure learning.
• To extend supervised covariance estimation to count data by integrating inverse covariance estimation into multivariate Poisson regression.
• To provide a computationally tractable method for high-dimensional multivariate count data with explicit modeling of conditional dependencies.

Proposed method

• Proposes a multivariate Poisson log-normal model where the log-rate parameters follow a multivariate normal distribution with a sparse inverse covariance matrix.
• Uses a Monte Carlo Expectation-Maximization (MCEM) algorithm to handle intractable likelihoods in parameter estimation.
• Applies Lasso penalty to the regression coefficient matrix to enforce sparsity and perform feature selection across multiple responses.
• Employs graphical Lasso on the latent variable covariance structure to estimate a sparse inverse covariance matrix, capturing conditional dependencies among responses.
• Alternates between M-step updates for the coefficient matrix and inverse covariance matrix using iterative reweighted least squares and coordinate descent.
• Derives a closed-form solution for the coefficient matrix update via vectorization and Kronecker product operations, enabling efficient computation.

Experimental results

Research questions

• RQ1 Can a joint multivariate Poisson log-normal model with sparse regression and precision matrix estimation improve prediction performance on multivariate count data compared to conventional methods?
• RQ2 How effective is the proposed MCEM algorithm in estimating both regression coefficients and the inverse covariance matrix in high-dimensional count data settings?
• RQ3 To what extent does incorporating conditional dependencies among count responses enhance model accuracy and feature selection?
• RQ4 Can the proposed method generalize to real-world multivariate count data, such as disease surveillance or traffic accident counts?
• RQ5 How does the performance of the model vary with different regularization penalties beyond Lasso, such as adaptive Lasso or group Lasso?

Key findings

• The proposed method significantly outperforms conventional univariate Poisson regression and multivariate normal regression on synthetic data, particularly in high-dimensional settings with complex dependencies.
• The model achieves improved prediction accuracy by jointly estimating regression coefficients and the inverse covariance matrix, capturing both marginal and conditional associations among responses.
• The use of Lasso and graphical Lasso penalties enables effective feature selection and identification of relevant predictors and conditional dependencies.
• Empirical results on an influenza-like illness dataset show that the model identifies meaningful sparsity patterns and accurately predicts multivariate count trends across regions.
• The MCEM algorithm converges reliably and efficiently, with computational complexity manageable even for moderate to large sample sizes and response dimensions.
• The method is extensible to other regularization penalties such as adaptive Lasso, group Lasso, or fused Lasso, maintaining strong performance across different sparsity structures.