[Paper Review] Inferences from prior-based loss functions

Michael Evans, Gun Ho Jang|arXiv (Cornell University)|Apr 16, 2011

Bayesian Modeling and Causal Inference16 references8 citations

TL;DR

This paper introduces prior-based loss functions that generate invariant, Bayesian-unbiased inferences through relative surprise ratios, offering a decision-theoretic foundation for relative surprise inferences. It proves that these inferences converge to the same limits as highest posterior density (HPD) regions and are invariant under reparameterization, providing a robust alternative to MAP-based methods.

ABSTRACT

Inferences that arise from loss functions determined by the prior are considered and it is shown that these lead to limiting Bayes rules that are closely connected with likelihood. The procedures obtained via these loss functions are invariant under reparameterizations and are Bayesian unbiased or limits of Bayesian unbiased inferences. These inferences serve as well-supported alternatives to MAP-based inferences.

Motivation & Objective

• To develop decision-theoretic foundations for relative surprise inferences that are invariant under reparameterization.
• To address the non-invariance of MAP-based inferences under smooth transformations of parameters.
• To establish connections between relative surprise inferences and Bayesian decision rules derived from prior-based loss functions.
• To show that relative surprise inferences are limits of Bayesian unbiased procedures and possess optimal frequentist properties.
• To provide a well-supported alternative to MAP estimation that maintains invariance and coherence under reparameterization.

Proposed method

• Defines a loss function derived from the prior density, leading to posterior risk minimization that induces relative surprise inferences.
• Constructs credible regions $ C_{ u}(x) = \{ \psi : \pi_{\Psi}(\psi|x)/\pi_{\Psi}(\psi) \geq c_{\nu}(x) \} $, where the ratio measures belief change from prior to posterior.
• Introduces a sequence of approximating regions $ C_{\lambda,\gamma}(x) $ using local neighborhoods to ensure convergence to the relative surprise region.
• Uses the limit of $ \Pi_{\Psi}(C_{\lambda,\gamma}(x)|x) $ as $ \lambda \to 0 $ to define the $ \gamma $-credible region, ensuring consistency with relative surprise principles.
• Applies convergence theorems to show that $ C_{\lambda,\gamma}(x) \to C_{\gamma}(x) $ in posterior probability as $ \lambda \to 0 $, establishing asymptotic equivalence.
• Demonstrates that the limiting regions $ C_{\gamma}(x) $ are equivalent to the relative surprise regions and inherit their invariance and optimality properties.

Experimental results

Research questions

• RQ1 Can prior-based loss functions generate inferences that are invariant under reparameterization?
• RQ2 How do relative surprise inferences based on prior-posterior density ratios compare to MAP-based inferences in terms of decision-theoretic optimality?
• RQ3 Do the resulting inferences from prior-based loss functions converge to well-defined limits that preserve Bayesian unbiasedness?
• RQ4 What is the relationship between the limiting regions from prior-based loss functions and the standard relative surprise regions?
• RQ5 Can these inferences be shown to be optimal in the class of all Bayesian inferences under appropriate decision-theoretic criteria?

Key findings

• The limiting Bayes rules derived from prior-based loss functions are invariant under reparameterization, resolving a key flaw of MAP estimation.
• The resulting inferences are equivalent to relative surprise regions, which are known to be optimal in terms of frequentist properties and invariance.
• The convergence $ C_{\lambda,\gamma}(x) \to C_{\gamma}(x) $ holds in posterior probability as $ \lambda \to 0 $, ensuring consistency of the approximation.
• The least relative surprise estimator (LRSE), defined as the maximizer of $ \pi_{\Psi}(\psi|x)/\pi_{\Psi}(\psi) $, is shown to be a limiting Bayes rule.
• The procedure yields credible regions that are both invariant and limits of Bayesian unbiased inferences, satisfying strong decision-theoretic criteria.
• The paper establishes that the limiting regions $ C_{\gamma}(x) $ are the same as those derived from relative surprise, confirming their optimality and coherence.