[Paper Review] Model comparison with missing data using MCMC and importance sampling

Panayiota Touloupou, Naif Alzahrani+3 more|arXiv (Cornell University)|Dec 15, 2015

Bayesian Methods and Mixture Models24 references3 citations

TL;DR

This paper proposes a computationally efficient method for estimating marginal likelihoods in Bayesian model comparison using a hybrid MCMC-importance sampling approach. By combining MCMC samples with an importance sampling estimator based on a parametric approximation to the posterior, the method outperforms existing techniques—including Chib’s method, power posteriors, and harmonic mean estimators—on epidemic and time series data, enabling robust model selection even for non-nested models where reversible jump MCMC fails.

ABSTRACT

Selecting between competing statistical models is a challenging problem especially when the competing models are non-nested. In this paper we offer a simple solution by devising an algorithm which combines MCMC and importance sampling to obtain computationally efficient estimates of the marginal likelihood which can then be used to compare the models. The algorithm is successfully applied to longitudinal epidemic and time series data sets and shown to outperform existing methods for computing the marginal likelihood.

Motivation & Objective

• To address the challenge of model comparison in Bayesian statistics when models are non-nested and reversible jump MCMC is infeasible.
• To develop a computationally efficient and robust method for estimating marginal likelihoods, especially when analytical computation is intractable.
• To enable accurate Bayes factor estimation for complex longitudinal and time series data with missing or incomplete observations.
• To provide a practical, automated algorithm that integrates MCMC, posterior approximation, and importance sampling for marginal likelihood estimation.

Proposed method

• The method uses MCMC to generate posterior samples from the model parameter space, approximating the true posterior with a parametric density q(θ).
• An importance sampling estimator is constructed using samples from q(θ), with weights derived from the ratio π(θ)/q(θ), to estimate the marginal likelihood π(x).
• The key estimator is given by ˆPq = (1/N) Σ π(x|θi) π(θi)/q(θi), where θi are drawn from q(θ), ensuring unbiased estimation of π(x).
• The approach is particularly effective for time series and longitudinal data, leveraging filtering techniques to compute π(x|θ) when analytical forms are unavailable.
• The algorithm is parallelizable, as importance sampling evaluations are independent, enabling scalable computation.
• The method avoids the need for reversible jump MCMC by using a consistent proposal distribution q(θ) that approximates the posterior, even when models differ structurally.

Experimental results

Research questions

• RQ1 Can a hybrid MCMC-importance sampling approach provide more accurate and efficient marginal likelihood estimates than existing methods in the presence of missing data and non-nested models?
• RQ2 How does the proposed method compare in performance to Chib’s method, power posteriors, and harmonic mean estimators for time series and epidemic data?
• RQ3 To what extent can the method support model selection when reversible jump MCMC is infeasible due to structural differences in competing models?
• RQ4 Can the method be effectively applied to real-world longitudinal data with complex dependence structures, such as seasonal trends and low-count time series?

Key findings

• The proposed method significantly outperformed Chib’s method, power posteriors, and harmonic mean estimators in estimating marginal likelihoods across multiple epidemic and time series datasets.
• For the pneumococcal transmission data, the method produced tighter Bayes Factor estimates than RJMCMC, with a log marginal likelihood of -263.13 for the model with seasonal covariates.
• In the cut injury data, the INAR(1) model with seasonal covariates had a log marginal likelihood of -286.0, compared to -298.3 for the standard INAR(1), indicating a substantial improvement.
• The Bayes Factor for the INAR(1) model with covariates over the standard INAR(1) was 2984, strongly favoring the model with seasonal effects.
• Posterior estimates showed that λ (infection intensity) varied significantly by season, while α (persistence) remained relatively stable, supporting seasonal transmission patterns.
• The method demonstrated robustness and computational efficiency, particularly in cases where RJMCMC failed due to structural differences between models.