[Paper Review] Bayesian finite mixtures: a note on prior specification and posterior computation

Agostino Nobile|ArXiv.org|Nov 3, 2007

Bayesian Methods and Mixture Models15 references15 citations

TL;DR

This paper proposes a novel method for computing the posterior distribution of the number of components in a Bayesian finite mixture model, advocating for a Poisson(1) prior on the number of components and providing a computationally efficient approach using MCMC output and marginal likelihood representations. The key contribution is a principled, scalable method for component estimation with validated performance on the galaxy data set.

ABSTRACT

A new method for the computation of the posterior distribution of the number k of components in a finite mixture is presented. Two aspects of prior specification are also studied: an argument is made for the use of a Poisson(1) distribution as the prior for k; and methods are given for the selection of hyperparameter values in the mixture of normals model, with natural conjugate priors on the components parameters.

Motivation & Objective

• To develop a reliable method for computing the posterior distribution of the number of components in Bayesian finite mixture models.
• To justify the use of a Poisson(1) prior for the number of components when no substantive prior information is available.
• To provide practical guidance for hyperparameter selection in the mixture of univariate normals model with natural conjugate priors.
• To improve the numerical computation of marginal likelihoods using MCMC output and probability identities.
• To demonstrate the method’s effectiveness using the galaxy data set as a benchmark.

Proposed method

• Uses a fundamental probability identity from Chib (1995) combined with marginal likelihood representations from Nobile (2004) to compute posterior distributions of the number of components.
• Employs MCMC sampling to estimate the frequency of empty components, which is used to approximate marginal likelihoods.
• Applies the representation of marginal likelihoods via allocation vectors and component-specific probabilities to derive computable expressions.
• Derives and uses recursive formulas (e.g., equations 14, 15) to compute ratios of marginal likelihoods across different component counts.
• Introduces a sensitivity analysis procedure based on median estimates of dispersion parameters (τ and δ) to guide hyperparameter selection.
• Validates the method using the galaxy data set and compares results across different values of k to assess model stability.

Experimental results

Research questions

• RQ1 What is the most appropriate non-informative prior for the number of components in a finite mixture model?
• RQ2 How can the marginal likelihood of a finite mixture model be efficiently computed from MCMC output?
• RQ3 What is the impact of hyperparameter choice on posterior inference in a mixture of normals model?
• RQ4 How can the posterior distribution of the number of components be reliably estimated in practice?
• RQ5 To what extent does the model structure naturally suggest a Poisson(1) prior for the number of components?

Key findings

• The Poisson(1) distribution is theoretically justified as a non-informative prior for the number of components due to its consistency with the model’s structural properties.
• The proposed method enables accurate and efficient computation of the posterior distribution of k using MCMC output and marginal likelihood approximations.
• Hyperparameter selection for the mixture of normals model is guided by a sensitivity analysis of posterior medians of τ and δ, suggesting τ levels off at k=4 and δ at k=6.
• The method yields stable estimates with τ̂ = 0.04 and δ̂ = 2, which are used effectively in subsequent MCMC runs.
• The recursive formulas (14) and (15) provide a computationally feasible way to compute ratios of marginal likelihoods across component counts.
• The approach is validated on the galaxy data set, showing robustness and practical utility in real-world applications.