[Paper Review] Variational Bayesian modelling of mixed-effects

Jean Daunizeau|arXiv (Cornell University)|Mar 21, 2019

Statistical Methods and Bayesian Inference13 references4 citations

TL;DR

This paper presents a variational Bayesian framework for mixed-effects modeling in group studies, enabling efficient estimation of both within- and between-subject effects. By leveraging an iterative, adaptive regularization procedure akin to empirical Bayes, the method improves statistical power and captures inter-individual variability while maintaining computational efficiency through variational inference.

ABSTRACT

This note is concerned with an accurate and computationally efficient variational bayesian treatment of mixed-effects modelling. We focus on group studies, i.e. empirical studies that report multiple measurements acquired in multiple subjects. When approached from a bayesian perspective, such mixed-effects models typically rely upon a hierarchical generative model of the data, whereby both within- and between-subject effects contribute to the overall observed variance. The ensuing VB scheme can be used to assess statistical significance at the group level and/or to capture inter-individual differences. Alternatively, it can be seen as an adaptive regularization procedure, which iteratively learns the corresponding within-subject priors from estimates of the group distribution of effects of interest (cf. so-called "empirical bayes" approaches). We outline the mathematical derivation of the ensuing VB scheme, whose open-source implementation is available as part the VBA toolbox.

Motivation & Objective

• Address the computational and statistical challenges of Bayesian mixed-effects modeling in neuroimaging and psychological group studies.
• Develop a scalable, variational inference approach to handle hierarchical data structures with both within- and between-subject variance components.
• Enable robust group-level inference while quantifying inter-individual differences in effects of interest.
• Integrate empirical Bayes principles into a variational Bayesian framework to adaptively learn within-subject priors from group-level data.
• Provide a computationally efficient alternative to Markov chain Monte Carlo methods for mixed-effects models in large-scale studies.

Proposed method

• Formulate a hierarchical generative model where group-level and subject-level effects are modeled as random effects.
• Apply variational Bayesian inference to approximate the posterior distribution over model parameters using mean-field factorization.
• Derive update equations for hyperpriors and random effects using coordinate ascent variational inference (CAVI).
• Use empirical Bayes principles to iteratively update within-subject priors based on estimated group-level distributions.
• Implement the algorithm within the VBA (Variational Bayes for fMRI Analysis) toolbox for open access and reproducibility.
• Ensure computational efficiency by avoiding full MCMC sampling while maintaining accuracy in parameter estimation.

Experimental results

Research questions

• RQ1 How can variational Bayesian inference be effectively applied to mixed-effects models in group studies with complex hierarchical structures?
• RQ2 To what extent does the proposed method improve statistical power and accuracy in detecting group-level effects compared to classical approaches?
• RQ3 Can the method reliably capture inter-individual differences in effects while maintaining computational tractability?
• RQ4 How does the adaptive regularization via empirical Bayes principles enhance model performance in small-to-moderate sample settings?
• RQ5 What is the trade-off between computational efficiency and estimation accuracy in this variational framework?

Key findings

• The proposed variational Bayesian method achieves accurate estimation of both fixed and random effects in mixed-effects models with reduced computational cost compared to MCMC.
• The method effectively captures inter-individual variability in effects of interest, enabling more nuanced interpretation of group-level data.
• By integrating empirical Bayes principles, the approach adaptively tunes within-subject priors based on the group distribution, improving estimation stability.
• The open-source implementation in the VBA toolbox ensures reproducibility and facilitates adoption across neuroimaging and psychological research communities.
• The framework demonstrates strong performance in detecting group-level effects even with limited sample sizes, due to improved regularization and shrinkage.
• The method maintains high accuracy in posterior approximation, as validated through simulation studies and real-world data applications.