[Paper Review] Bayesian Multinomial Logistic Normal Models through Marginally Latent Matrix-T Processes

Justin D. Silverman, Kimberly Roche+3 more|arXiv (Cornell University)|Mar 27, 2019

Bayesian Methods and Mixture Models29 references40 citations

TL;DR

The paper introduces Marginally Latent Matrix-T Processes to enable fast, scalable Bayesian inference for multinomial logistic-normal models, with a collapse-uncollapse (CU) sampler and Laplace approximation that greatly improves efficiency over traditional MCMC approaches.

ABSTRACT

Bayesian multinomial logistic-normal (MLN) models are popular for the analysis of sequence count data (e.g., microbiome or gene expression data) due to their ability to model multivariate count data with complex covariance structure. However, existing implementations of MLN models are limited to handling small data sets due to the non-conjugacy of the multinomial and logistic-normal distributions. We introduce MLN models which can be written as marginally latent matrix-t process (LTP) models. Marginally LTP models describe a flexible class of generalized linear regression, non-linear regression, and time series models. We develop inference schemes for Marginally LTP models and, through application to MLN models, demonstrate that our inference schemes are both highly accurate and often 4-5 orders of magnitude faster than MCMC.

Motivation & Objective

• Motivate efficient Bayesian inference for multinomial logistic-normal (MLN) models used on multivariate count data.
• Introduce Marginally Latent Matrix-T Process (Marginally LTP) as a generalizable framework encompassing MLN models.
• Develop a Collapse-Uncollapse (CU) sampler to sample from collapsed LTP form and then uncollapse to the full model.
• Provide a Laplace approximation to accelerate sampling from the collapsed form and demonstrate scalability to large D and N.
• Offer software implementation (R package stray) and reproducible results for simulations and microbiome data.

Proposed method

• Define MLN linear models with additive log-ratio transformation and matrix-normal priors.
• Introduce Marginally Latent Matrix-T Process (Marginally LTP) as a latent representation that marginalizes over latent matrices to obtain a matrix-T process.
• Develop a Collapse-Uncollapse (CU) sampler that alternates sampling from p(eta|Y) and p(Psi|eta,Y).
• Derive a Laplace approximation q(eta|Y) using MAP and Hessian of log p(eta|Y) to accelerate collapsed-form sampling.
• Show how p(Psi|eta,Y) reduces to standard Bayesian Gaussian models for GMCL and GMDLM subsets.
• Provide software stray and open-source code for reproducing results.

Experimental results

Research questions

• RQ1 How can MLN models be made scalable to hundreds or thousands of categories and samples without resorting to low-rank or windowed approaches?
• RQ2 Can Marginally LTPs provide accurate and fast posterior inference for MLN and related models across generalized linear and time-series settings?
• RQ3 Does a collapse-uncollapse inference scheme with a Laplace approximation match or outperform existing MCMC/VB approaches in accuracy and efficiency?
• RQ4 What is the impact of using Laplace-approximated collapsed sampling on inference quality in high-dimensional count data models?

Key findings

• The Marginally LTP framework subsumes GMCL and GMDLM as special cases, enabling broad modeling flexibility.
• The CU sampler, which first samples from the collapsed LTP p(eta|Y) and then samples p(Psi|eta,Y), achieves substantial computational gains over full MCMC in MLN contexts.
• A Laplace approximation to p(eta|Y) yields further speedups with accurate posterior approximations for the collapsed form.
• Across simulations, the CU sampler with Laplace approximation demonstrated high accuracy and improved efficiency (relative to HMC Uncollapsed and VB variants) for MLN models.
• The authors provide the stray R package and GitHub code to reproduce simulations and microbiome data analyses.