[Paper Review] Asymptotic Approximation of Marginal Likelihood Integrals

Shaowei Lin|arXiv (Cornell University)|Mar 28, 2010

Polynomial and algebraic computation23 references16 citations

TL;DR

This paper presents an asymptotic method for approximating marginal likelihood integrals in Bayesian statistics by reducing the problem to computing the real log canonical threshold (RLCT) of a polynomial ideal via computational algebraic geometry. Using resolution of singularities, the approach enables accurate evaluation of marginal likelihoods for all discrete exponential family models with real analytic parametrizations, providing a systematic and effective solution to a long-standing problem in Bayesian model comparison.

ABSTRACT

The accurate asymptotic evaluation of marginal likelihood integrals is a fundamental problem in Bayesian statistics. Following the approach introduced by Watanabe, we translate this into a problem of computational algebraic geometry, namely, to determine the real log canonical threshold of a polynomial ideal, and we present effective methods for solving this problem. Our results are based on resolution of singularities, and they apply to all statistical models for discrete data that admit a parametrization by real analytic functions.

Motivation & Objective

• To address the fundamental challenge of asymptotically evaluating marginal likelihood integrals in Bayesian model comparison.
• To translate the problem of marginal likelihood evaluation into a computational algebraic geometry problem involving polynomial ideals.
• To develop effective algorithms for computing the real log canonical threshold (RLCT), a key quantity in asymptotic marginal likelihood approximation.
• To extend the applicability of asymptotic methods to all discrete statistical models with real analytic parametrizations.

Proposed method

• The method reduces the marginal likelihood integral to the computation of the real log canonical threshold (RLCT) of a polynomial ideal derived from the model's likelihood and prior.
• It employs resolution of singularities to analyze the singularities of the likelihood-prior product, enabling precise asymptotic expansion of the integral.
• The approach leverages algebraic geometry tools to compute the RLCT without requiring explicit integration, relying instead on geometric and algebraic invariants.
• The algorithmic framework is general and applies to all discrete exponential family models with real analytic parameterizations.
• The method ensures convergence and accuracy in the large-sample limit by exploiting the structure of the singularities in the parameter space.
• The solution is implemented through symbolic computation techniques that extract the RLCT from the resolution data.

Experimental results

Research questions

• RQ1 How can marginal likelihood integrals be asymptotically approximated in a way that is both accurate and computationally feasible for general discrete models?
• RQ2 What algebraic-geometric invariant governs the asymptotic behavior of marginal likelihoods in Bayesian models?
• RQ3 Can the real log canonical threshold (RLCT) be effectively computed for all real analytic statistical models?
• RQ4 How does resolution of singularities enable the derivation of asymptotic expansions for marginal likelihoods?
• RQ5 What is the connection between the geometry of the model's parameter space and the asymptotic marginal likelihood?

Key findings

• The real log canonical threshold (RLCT) is the central quantity determining the asymptotic behavior of marginal likelihood integrals in Bayesian models.
• The RLCT can be computed algorithmically using resolution of singularities, providing a systematic method for asymptotic marginal likelihood evaluation.
• The method applies universally to all discrete statistical models with real analytic parametrizations, including exponential families.
• The asymptotic approximation of the marginal likelihood is governed by the RLCT and the sample size, with convergence rates determined by the singularity structure.
• The approach enables accurate model comparison in high-dimensional and singular models where standard Laplace approximations fail.