[Paper Review] Statistical Guarantees for Data-driven Posterior Tempering

Ruchira Ray, Marco Avella Medina+1 more|arXiv (Cornell University)|Jan 14, 2026

Fractional Differential Equations Solutions0 citations

TL;DR

This paper analyzes data-driven tempering in power posteriors (alpha-posterior), establishing consistency and Bernstein–von Mises type results under regimes where alpha is random and n-dependent, and introduces a new Laplace approximation crucial for these results.

ABSTRACT

Posterior tempering reduces the influence of the likelihood in the calculation of the posterior by raising the likelihood to a fractional power $α$. The resulting power posterior - also known as an $α$-posterior or fractional posterior - has been shown to exhibit appealing properties, including robustness to model misspecification and asymptotic normality (Bernstein-von Mises theorem). However, practical recommendations for selecting the tempering parameter and statistical guarantees for the resulting power posterior remain open questions. Cross-validation-based approaches to tuning this parameter suggest interesting asymptotic regimes for the selected $α$, which can either vanish or behave like a mixture distribution with a point mass at infinity and the remaining mass converging to zero. We formalize the asymptotic properties of the power posterior in these regimes. In particular, we provide sufficient conditions for (i) consistency of the power posterior moments and (ii) asymptotic normality of the power posterior mean. Our analysis required us to establish a new Laplace approximation that is interesting in its own right and is the key technical tool for showing a critical threshold $α\asymp 1/\sqrt{n}$ where the asymptotic normality of the posterior mean breaks. Our results allow for the power to depend on the data in an arbitrary way.

Motivation & Objective

• Motivate and formalize data-driven tempering via alpha-posteriors as a robust alternative under model misspecification.
• Characterize asymptotic regimes for alpha_n that arise from cross-validation and other tuning methods.
• Establish conditions for consistency of alpha_n-posterior moments and for asymptotic normality of the alpha_n-posterior mean.
• Develop a new Laplace approximation to analyze the impact of data-dependent tempering and identify critical thresholds for normality.

Proposed method

• Define alpha-posterior: pi_{n,alpha}( heta|X^n) ∝ f_n(X^n|θ)^α π(θ).
• Study regimes where alpha_n satisfies 1/n << alpha_n << 1 and derive moment consistency (Theorem 1).
• Prove Bernstein–von Mises type results for alpha_n-posterior moments and for the alpha_n-posterior mean (Corollaries 1 and 2).
• Develop a novel Laplace approximation (Lemma 1) to quantify the distance between posterior mean and MLE and to identify the threshold alpha_n ~ 1/√n.
• Analyze mixed regimes where alpha_n either tends to infinity with positive probability or vanishes (Theorem 3).
• Empirically investigate data-driven alpha_n tuning methods (BCV, BCV+VI, LOOCV, Train-test, SafeBayes) and illustrate with simulations and CPS1988 data.

Experimental results

Research questions

• RQ1 What are the asymptotic properties (consistency, normality) of alpha_n-posteriors when alpha_n is random and data-dependent?
• RQ2 What are the necessary and sufficient conditions for Bernstein–von Mises behavior and for the asymptotic normality of the alpha_n-posterior mean in data-driven tempering regimes?
• RQ3 How does data-driven tuning of alpha_n (via cross-validation, SafeBayes, etc.) behave asymptotically (e.g., vanishing, infinity with mass, or mixed regimes)?
• RQ4 What is the role of a new Laplace approximation in establishing these asymptotic properties and identifying critical thresholds (e.g., alpha ~ 1/√n)?

Key findings

• Consistency of alpha_n-posterior moments in the regime 1/n << alpha_n << 1, with moments converging to those of a normal distribution centered at the MLE.
• Bernstein–von Mises type results hold for the alpha_n-posterior under the same regime, with sharp conditions showing 1/n << alpha_n is necessary for BvM.
• Asymptotic normality of the alpha_n-posterior mean established when 1/√n << alpha_n << 1, proven via a new Laplace approximation (Lemma 1).
• In the regime alpha_n → ∞ in probability, the alpha_n-posterior concentrates at the MLE (point mass at the MLE).
• In mixed regimes where alpha_n either diverges with positive probability or vanishes, a BvM-type result extends to a mixed setting (Theorem 3).
• Numerical and real-data experiments show data-driven alpha_n can vanish, concentrate at infinity, or exhibit a mixed distribution, motivating the theoretical regimes.