[Paper Review] A latent variable model with mixed binary and continuous response variables

Klaus K. Holst, Esben Budtz–Jørgensen+1 more|arXiv (Cornell University)|Jul 5, 2015

Statistical Methods and Bayesian Inference5 references3 citations

TL;DR

This paper proposes a latent variable model that jointly estimates continuous, binary, and censored outcomes using a Probit link and multivariate normal latent variables, enabling efficient maximum likelihood estimation via orthant probability computations. The method outperforms limited information estimators in small samples and is implemented in the R package lava.tobit, offering a scalable alternative to quadrature-based methods for complex structural equation models.

ABSTRACT

We propose a method for obtaining maximum likelihood estimates in a model with continuous and binary outcomes. Combinations of left and right censored observations are also naturally modeled in this framework. The model and estimation procedure has been implemented in the R package lava.tobit. The method is demonstrated on brain imaging and personality data where measurement error on predictor variables is handled in a latent variable framework. A simulation study is conducted comparing the small sample properties of the MLE with a limited information estimator.

Motivation & Objective

• To develop a maximum likelihood estimation framework for structural equation models with mixed response types: continuous, binary, and censored outcomes.
• To address the limitations of existing methods like AGQ and MCEM, which suffer from high computational cost or convergence issues in high-dimensional latent variable settings.
• To enable estimation of conditional (subject-specific) effects and variance components under a latent variable framework with measurement error correction.
• To provide a computationally efficient alternative to adaptive Gaussian quadrature and Monte Carlo EM by leveraging multivariate normal orthant probabilities.
• To demonstrate the method on neuroimaging and personality data, showing its utility in modeling complex relationships between biological markers and psychological traits.

Proposed method

• The model uses a latent variable framework where binary outcomes are generated via a threshold mechanism on underlying normally distributed latent variables.
• Continuous outcomes are modeled directly as the latent variable, while censored outcomes are handled through truncation of the latent variable distribution.
• The likelihood function is derived from multivariate normal orthant probabilities, avoiding numerical integration over random effects.
• The score function and observed information matrix are computed analytically using derivatives of orthant probabilities, enabling fast and precise MLE.
• The method supports random slopes via interaction terms between covariates and random effects, allowing for heterogeneous effects across individuals.
• The approach is implemented in the open-source R package lava.tobit, which supports full model specification including non-linear covariate effects through parametric functions.

Experimental results

Research questions

• RQ1 How can a unified latent variable model efficiently handle mixed continuous, binary, and censored outcomes in a single estimation framework?
• RQ2 What are the small-sample properties of maximum likelihood estimation in this model compared to limited information estimators?
• RQ3 Can the use of orthant probabilities for multivariate normal distributions replace computationally intensive numerical integration in high-dimensional latent variable models?
• RQ4 How does the method perform in estimating conditional effects and variance components in the presence of measurement error in predictors?
• RQ5 To what extent can this model support causal inference in complex structural equation models with non-normal outcomes?

Key findings

• The proposed maximum likelihood estimator demonstrated superior small-sample performance compared to the limited information estimator, particularly in terms of bias and coverage probability.
• The method achieved a standardized regression coefficient of 0.791 (p-value 0.04) for the effect of 5-HT2A receptor binding potential on the latent personality trait Dependence, indicating a strong association.
• The probability of a positive response to a personality questionnaire item increased by 20% (95% CI: [1.05, 1.34]) for individuals with 5-HT2A receptor binding two standard deviations above the mean.
• The model successfully handled measurement error in predictors and allowed for conditional probability estimation of item responses given latent trait levels.
• The algorithm scaled efficiently with model complexity, avoiding the curse of dimensionality inherent in AGQ and MCEM methods.
• Numerical instability was observed when including all items in the measurement model, requiring removal of one item (question 131) to ensure convergence.