[Paper Review] Conditional Inference for Multivariate Generalised Linear Mixed Models

Jeanett S. Pelck, Rodrigo Labouriau|arXiv (Cornell University)|Jul 25, 2021

Statistical Methods and Bayesian Inference13 references4 citations

TL;DR

This paper proposes a novel conditional inference method for multivariate generalised linear mixed models (MGLMMs) that avoids integrating likelihoods by predicting random effects. It extends GLMMs to non-Gaussian random effects (e.g., multivariate t-distributions) and dispersion models, enabling flexible modeling of diverse response types with complex dependence structures while maintaining asymptotic efficiency and computational feasibility.

ABSTRACT

We propose a method for inference in generalised linear mixed models (GLMMs) and several extensions of these models. First, we extend the GLMM by allowing the distribution of the random components to be non-Gaussian, that is, assuming an absolutely continuous distribution with respect to the Lebesgue measure that is symmetric around zero, unimodal and with finite moments up to fourth-order. Second, we allow the conditional distribution to follow a dispersion model instead of exponential dispersion models. Finally, we extend these models to a multivariate framework where multiple responses are combined by imposing a multivariate absolute continuous distribution on the random components representing common clusters of observations in all the marginal models. Maximum likelihood inference in these models involves evaluating an integral that often cannot be computed in closed form. We suggest an inference method that predicts values of random components and does not involve the integration of conditional likelihood quantities. The multivariate GLMMs that we studied can be constructed with marginal GLMMs of different statistical nature, and at the same time, represent complex dependence structure providing a rather flexible tool for applications.

Motivation & Objective

• To develop a likelihood-free inference method for GLMMs that avoids high-dimensional integration of conditional likelihoods.
• To extend standard GLMMs by allowing non-Gaussian, symmetric, unimodal random effects with finite fourth moments.
• To generalize the conditional distribution family beyond exponential dispersion models to include broader dispersion models.
• To construct multivariate GLMMs where marginal models can be of different statistical types (e.g., binomial, Poisson, Gamma) while sharing a common multivariate random effect structure.
• To provide a computationally efficient and asymptotically valid inference framework applicable to complex, real-world multivariate data with heterogeneous responses.

Proposed method

• Uses inference functions based on conditional estimating equations that bypass the need to integrate over random effects.
• Predicts random effects values directly using a conditional estimation approach, avoiding numerical integration of the likelihood.
• Employs a multivariate absolute continuous distribution (e.g., multivariate normal or t) for the random effects, allowing flexible dependence structures across clusters.
• Extends the model framework to include dispersion models instead of restricting to exponential dispersion families, increasing model flexibility.
• Applies a multivariate extension of the Laplace approximation in Appendix A.4 for comparison and validation of the proposed method.
• Uses Hermite approximation and simulation-based evaluation to assess performance across different random effect distributions and cluster sizes.

Experimental results

Research questions

• RQ1 Can a conditional inference method be developed for GLMMs that avoids integrating over random effects while preserving asymptotic efficiency?
• RQ2 How do non-Gaussian random effects (e.g., multivariate t-distribution) affect the performance and robustness of inference in multivariate GLMMs?
• RQ3 To what extent can marginal models in MGLMMs differ in distributional family and link function while still maintaining a coherent multivariate dependence structure?
• RQ4 How does the proposed inference method compare to existing methods like Breslow and Clayton’s (1993) Laplace approximation in terms of bias, standard error, and normality of estimators?
• RQ5 Can the method be extended to dispersion models beyond exponential families, and what are the implications for model flexibility and inference accuracy?

Key findings

• The proposed conditional inference method achieves performance comparable to Breslow and Clayton’s (1993) Laplace approximation when random effects are Gaussian, with similar bias and standard errors.
• The method maintains good finite-sample properties even when random effects follow heavy-tailed distributions such as the multivariate t-distribution.
• Simulation studies show that the sampling distribution of parameter estimates is approximately normal, with p-values from Shapiro-Wilk tests ranging from 0.15 to 0.85, indicating good normality approximation.
• Bias in parameter estimates remains low across different cluster sizes (q = 10, 50, 100), with estimated bias values below 0.05 in absolute terms for most parameters.
• The method successfully handles multivariate GLMMs with mixed marginal families (e.g., binomial, Poisson, Gamma), enabling joint modeling of heterogeneous responses.
• The multivariate extension of the Laplace approximation in Appendix A.4 confirms the consistency of the proposed inference framework under standard regularity conditions.