[Paper Review] On posterior propriety for the Student-$t$ linear regression model under Jeffreys priors

Catalina A. Vallejos, Mark F. J. Steel|arXiv (Cornell University)|Nov 6, 2013

Statistical Methods and Bayesian Inference8 references3 citations

TL;DR

This paper corrects a critical error in prior work on Bayesian inference for Student-$t$ linear regression under Jeffreys priors. It demonstrates that the standard Jeffreys-rule prior leads to an improper posterior distribution, rendering Bayesian inference invalid, while the independence Jeffreys prior ensures posterior propriety and is therefore recommended for practical use.

ABSTRACT

Regression models with fat-tailed error terms are an increasingly popular choice to obtain more robust inference to the presence of outlying observations. This article focuses on Bayesian inference for the Student-$t$ linear regression model under objective priors that are based on the Jeffreys rule. Posterior propriety results presented in Fonseca et al. (2008) are revisited and corrected. In particular, it is shown that the standard Jeffreys-rule prior precludes the existence of a proper posterior distribution.

Motivation & Objective

• To re-examine posterior propriety in Bayesian Student-$t$ linear regression under Jeffreys priors.
• To identify and correct an error in prior claims about posterior existence under the Jeffreys-rule prior.
• To clarify which objective priors yield proper posteriors for the degrees of freedom parameter $\nu$.
• To provide a rigorous condition for posterior propriety based on the prior's dependence on $\nu$.
• To guide practitioners toward valid objective priors for robust regression with heavy-tailed errors.

Proposed method

• Derives the Fisher information matrix for the Student-$t$ linear regression model with parameters $\beta$, $\sigma^2$, and $\nu$.
• Constructs two objective priors using Jeffreys' rule: the standard Jeffreys-rule prior and the independence Jeffreys prior.
• Applies Fubini's theorem and marginalization techniques to integrate out $\beta$ and $\sigma^2$, focusing on the posterior's dependence on $\nu$.
• Uses order statistics and bounds on gamma-type integrals to derive necessary conditions for posterior propriety.
• Applies a result from Fernández and Steel (1999) to confirm posterior propriety under the independence Jeffreys prior.
• Derives a necessary condition for posterior propriety: $\nu > \frac{2a - 2}{n - p}$, where $a$ is the exponent in the $\sigma^2$-component of the prior.

Experimental results

Research questions

• RQ1 Does the standard Jeffreys-rule prior for the Student-$t$ linear regression model yield a proper posterior distribution?
• RQ2 What is the necessary condition for posterior propriety under Jeffreys priors in this model?
• RQ3 Why do previous studies, including Fonseca et al. (2008), incorrectly claim posterior propriety under the Jeffreys-rule prior?
• RQ4 Can the independence Jeffreys prior be used reliably for Bayesian inference in this model?
• RQ5 How does the impropriety of the Jeffreys-rule prior relate to poor frequentist coverage observed in small samples?

Key findings

• The standard Jeffreys-rule prior for the Student-$t$ linear regression model results in an improper posterior distribution, invalidating Bayesian inference.
• The necessary condition for posterior propriety is $\nu > \frac{2a - 2}{n - p}$, which is violated when $a > 1$ and $\nu$ is allowed over $ (0, \infty) $.
• For the Jeffreys-rule prior, $a = 1 + p/2 > 1$, so the posterior is improper, contradicting Fonseca et al. (2008).
• The independence Jeffreys prior, with $a = 1$, satisfies the necessary condition and ensures a proper posterior distribution.
• Posterior propriety under the independence Jeffreys prior is guaranteed for $n > p$, as confirmed by Theorem 1 in Fernández and Steel (1999).
• The impropriety of the Jeffreys-rule prior may explain the poor frequentist coverage of 95% credible intervals for $\nu$ in small samples, as observed in prior studies.