[Paper Review] Comment on "Bayesian evidence: can we beat MultiNest using traditional MCMC methods", by Rutger van Haasteren (arXiv:0911.2150)

F. Feroz, M. P. Hobson+1 more|arXiv (Cornell University)|Jan 5, 2010

Gaussian Processes and Bayesian Inference30 references3 citations

TL;DR

This paper critiques Rutger van Haasteren's claim that a Voronoi tessellation-based method outperforms MultiNest in Bayesian evidence evaluation. The authors argue the comparison is fundamentally unfair because van Haasteren's method relies on pre-existing posterior samples, while MultiNest performs end-to-end Bayesian inference—including posterior exploration and evidence calculation—without such assumptions, making it more robust and generalizable for complex, multi-modal problems.

ABSTRACT

In arXiv:0911.2150, Rutger van Haasteren seeks to criticize the nested sampling algorithm for Bayesian data analysis in general and its MultiNest implementation in particular. He introduces a new method for evidence evaluation based on the idea of Voronoi tessellation and requiring samples from the posterior distribution obtained through MCMC based methods. He compares its accuracy and efficiency with MultiNest, concluding that it outperforms MultiNest in several cases. This comparison is completely unfair since the proposed method can not perform the complete Bayesian data analysis including posterior exploration and evidence evaluation on its own while MultiNest allows one to perform Bayesian data analysis end to end. Furthermore, their criticism of nested sampling (and in turn MultiNest) is based on a few conceptual misunderstandings of the algorithm. Here we seek to set the record straight.

Motivation & Objective

• To challenge the validity of van Haasteren's claim that his Voronoi tessellation method outperforms MultiNest in Bayesian evidence evaluation.
• To clarify conceptual misunderstandings in van Haasteren's critique of nested sampling and MultiNest.
• To emphasize that MultiNest provides a complete, self-contained solution for Bayesian inference, unlike methods requiring external posterior samples.
• To demonstrate that MultiNest accurately computes evidence even in high-dimensional, multi-modal problems where other methods fail.
• To reaffirm the superiority of nested sampling and MultiNest for complex inference tasks in cosmology and astrophysics.

Proposed method

• The authors analyze van Haasteren's method, which uses Voronoi tessellation on posterior samples to estimate Bayesian evidence.
• They highlight that the method is not self-contained and depends on prior posterior sampling via MCMC, which MultiNest performs internally.
• The critique centers on the conceptual error of comparing a post-processing tool (van Haasteren's method) with a full inference pipeline (MultiNest).
• The authors use a high-dimensional ellipsoidal Gaussian likelihood with uniform priors to test MultiNest’s evidence accuracy across dimensions 2 to 32.
• They compute the log-evidence and information content (H) using MultiNest with live points and adaptive efficiency settings.
• The analytical log-evidence is 0.0 for all dimensions, serving as a benchmark for accuracy.

Experimental results

Research questions

• RQ1 Can a method based on posterior samples via MCMC outperform MultiNest in Bayesian evidence evaluation?
• RQ2 Is the comparison between van Haasteren’s Voronoi-based method and MultiNest fair and valid?
• RQ3 Does MultiNest accurately compute Bayesian evidence in high-dimensional, multi-modal, and degenerate parameter spaces?
• RQ4 What are the limitations of post-processing methods that rely on external posterior samples compared to end-to-end inference tools?
• RQ5 How does the performance of MultiNest degrade with increasing dimensionality in evidence evaluation?

Key findings

• MultiNest accurately computes the Bayesian evidence with log(Z) ≈ 0.0 across dimensions 2 to 32, matching the analytical value within error bounds.
• For the 32D problem, MultiNest required 14.1 million likelihood evaluations and achieved log(Z) = 0.40 ± 0.31, showing acceptable accuracy despite high dimensionality.
• The information content H increased to 96.10 at 32D, confirming the exponential increase in complexity, yet MultiNest remained effective.
• MultiNest maintains accuracy even when the posterior occupies only e^−96.10 of the prior volume, demonstrating robustness in high-dimensional, low-likelihood regions.
• The authors confirm that MultiNest can compute evidence accurately up to ~1000D using Hamiltonian sampling, indicating strong scalability.
• The comparison in van Haasteren (2009) is invalid because it pits a tool requiring posterior samples against a method that generates them internally, making the evaluation inherently unfair.