[Paper Review] Bayes Factors for Peri-Null Hypotheses

Alexander Ly, Eric‐Jan Wagenmakers|arXiv (Cornell University)|Feb 14, 2021

Statistical Methods and Inference4 citations

TL;DR

This paper investigates the use of peri-null hypotheses—narrow priors centered on a point-null value—as a practical alternative to sharp point-null hypotheses in Bayesian hypothesis testing. It shows that while peri-null Bayes factors are nearly identical to point-null Bayes factors for moderate sample sizes, they become inconsistent for large samples, converging to a limit determined by the ratio of prior densities at the maximum likelihood estimate, undermining their reliability under the alternative hypothesis.

ABSTRACT

A perennial objection against Bayes factor point-null hypothesis tests is that the point-null hypothesis is known to be false from the outset. We examine the consequences of approximating the sharp point-null hypothesis by a hazy `peri-null' hypothesis instantiated as a narrow prior distribution centered on the point of interest. The peri-null Bayes factor then equals the point-null Bayes factor multiplied by a correction term which is itself a Bayes factor. For moderate sample sizes, the correction term is relatively inconsequential; however, for large sample sizes the correction term becomes influential and causes the peri-null Bayes factor to be inconsistent and approach a limit that depends on the ratio of prior ordinates evaluated at the maximum likelihood estimate. We characterize the asymptotic behavior of the peri-null Bayes factor and briefly discuss suggestions on how to construct peri-null Bayes factor hypothesis tests that are also consistent.

Motivation & Objective

• To address the common criticism that point-null hypotheses are always false in practice by replacing them with peri-null hypotheses.
• To examine the asymptotic behavior of Bayes factors when the null hypothesis is approximated by a narrow prior distribution centered on the point-null value.
• To evaluate whether peri-null Bayes factors maintain consistency—i.e., correctly favor the true hypothesis as sample size increases—under both the null and alternative models.
• To propose and analyze alternative constructions of peri-null priors that restore consistency in large-sample settings.
• To clarify the implications of prior width choice on the asymptotic behavior of Bayes factors in hypothesis testing.

Proposed method

• The paper models the peri-null hypothesis as a narrow prior distribution centered on the point-null value, specifically using a normal distribution with small variance κ₀².
• It derives the peri-null Bayes factor as the product of the standard point-null Bayes factor and a correction term, which is itself a Bayes factor comparing the peri-null to the point-null prior.
• The correction term is shown to depend on the ratio of prior densities evaluated at the maximum likelihood estimate, which becomes influential under large sample sizes.
• Asymptotic sampling distributions of the peri-null Bayes factor are derived using Laplace approximations, revealing convergence to a limit that depends on prior density ratios.
• Three alternative constructions are analyzed: the fixed-width peri-null, the peri-point mixture model, and the shrinking peri-null (where κ₀ decreases with n).
• Theoretical consistency is assessed by examining whether the Bayes factor tends to infinity under the null and to zero under the alternative as n → ∞.

Experimental results

Research questions

• RQ1 Does replacing a point-null hypothesis with a peri-null hypothesis (a narrow prior around the null value) preserve the consistency of Bayes factors in large samples?
• RQ2 How does the asymptotic behavior of the peri-null Bayes factor differ from that of the standard point-null Bayes factor?
• RQ3 What is the role of the prior width (κ₀) in determining the limiting value of the peri-null Bayes factor?
• RQ4 Can alternative peri-null constructions—such as shrinking widths or mixture priors—restore consistency to the Bayes factor?
• RQ5 To what extent does the correction factor, arising from the prior density ratio at the MLE, dominate the behavior of the peri-null Bayes factor in large samples?

Key findings

• For moderate sample sizes, the peri-null Bayes factor is nearly identical to the point-null Bayes factor, making the approximation practically unproblematic.
• For large sample sizes, the peri-null Bayes factor becomes inconsistent, converging to a finite limit rather than diverging under the null or converging to zero under the alternative.
• The limiting value of the peri-null Bayes factor is determined by the ratio of prior densities at the maximum likelihood estimate, which is independent of the data and depends only on the prior width and model parameters.
• The peri-point mixture model achieves consistency only under the null hypothesis, not under the alternative, and thus fails to satisfy the standard consistency desideratum.
• The shrinking peri-null hypothesis (e.g., κ₀ = cσ/√n) restores consistency but introduces incoherence, as the prior depends on the intended sample size, violating the likelihood principle.
• The paper concludes that the peri-null approach offers little advantage over the point-null in terms of asymptotic behavior and may lead to misleading conclusions due to bounded evidence under the alternative.