[Paper Review] The Inverse Gamma-Gamma Prior for Optimal Posterior Contraction and Multiple Hypothesis Testing

Ray Bai, Malay Ghosh|arXiv (Cornell University)|Oct 12, 2017

Statistical Methods and Inference8 references7 citations

TL;DR

This paper introduces the inverse gamma-gamma (IGG) prior for Bayesian estimation of sparse high-dimensional mean vectors under Gaussian noise. By leveraging a scale-mixture normal structure, the IGG achieves near-minimax posterior contraction and outperforms the horseshoe and horseshoe+ in Kullback-Leibler concentration; it also enables a hypothesis test that asymptotically attains the optimal Bayes risk for signal classification.

ABSTRACT

We study the well-known problem of estimating a sparse $n$-dimensional unknown mean vector $θ= (θ_1, ..., θ_n)$ with entries corrupted by Gaussian white noise. In the Bayesian framework, continuous shrinkage priors which can be expressed as scale-mixture normal densities are popular for obtaining sparse estimates of $θ$. In this article, we introduce a new fully Bayesian scale-mixture prior known as the inverse gamma-gamma (IGG) prior. We prove that the posterior distribution contracts around the true $θ$ at (near) minimax rate under very mild conditions. In the process, we prove that the sufficient conditions for minimax posterior contraction given by Van der Pas et al. (2016) are not necessary for optimal posterior contraction. We further show that the IGG posterior density concentrates at a rate faster than those of the horseshoe or the horseshoe+ in the Kullback-Leibler (K-L) sense. To classify true signals ($θ_i eq 0$), we also propose a hypothesis test based on thresholding the posterior mean. Taking the loss function to be the expected number of misclassified tests, we show that our test procedure asymptotically attains the optimal Bayes risk exactly. We illustrate through simulations and data analysis that the IGG has excellent finite sample performance for both estimation and classification.

Motivation & Objective

• To develop a new Bayesian prior that enables optimal posterior contraction for sparse high-dimensional mean vectors.
• To improve upon existing shrinkage priors like the horseshoe and horseshoe+ in terms of posterior concentration and frequentist optimality.
• To design a hypothesis testing procedure based on the posterior mean that asymptotically achieves the optimal Bayes risk for multiple testing.
• To establish theoretical conditions under which posterior contraction is optimal, even when prior conditions from Van der Pas et al. (2016) are not satisfied.

Proposed method

• Proposes the inverse gamma-gamma (IGG) prior as a scale-mixture of normals, with a hierarchical structure involving inverse gamma and gamma distributions.
• Derives the posterior distribution under the IGG prior and establishes its contraction properties using frequentist risk criteria.
• Uses the Kullback-Leibler divergence to compare the concentration rates of the IGG posterior with those of the horseshoe and horseshoe+.
• Develops a multiple hypothesis testing procedure based on thresholding the posterior mean, minimizing expected misclassification loss.
• Employs theoretical analysis to show that the proposed test attains the optimal Bayes risk asymptotically.
• Validates performance through simulations and real data analysis, demonstrating strong finite-sample behavior.

Experimental results

Research questions

• RQ1 Can a new scale-mixture prior be constructed that achieves near-minimax posterior contraction under weaker conditions than existing results?
• RQ2 How does the posterior concentration rate of the IGG prior compare to that of the horseshoe and horseshoe+ in the Kullback-Leibler sense?
• RQ3 Can a Bayesian multiple testing procedure based on the IGG posterior mean achieve the optimal Bayes risk asymptotically?
• RQ4 Are the sufficient conditions for minimax posterior contraction from Van der Pas et al. (2016) necessary, or can optimality be achieved under weaker assumptions?

Key findings

• The IGG prior achieves posterior contraction at a (near) minimax rate under very mild regularity conditions, even when the conditions from Van der Pas et al. (2016) are not satisfied.
• The IGG posterior concentrates faster than both the horseshoe and horseshoe+ in the Kullback-Leibler divergence, indicating superior posterior concentration.
• The proposed hypothesis test based on the posterior mean asymptotically attains the optimal Bayes risk, meaning it minimizes the expected number of misclassified tests exactly in the limit.
• Simulations and data analysis confirm that the IGG prior delivers excellent finite-sample performance in both estimation and multiple hypothesis testing.
• The theoretical framework shows that the sufficient conditions for optimal posterior contraction are not necessary, broadening the scope for constructing effective shrinkage priors.