[Paper Review] Efficient Empirical Bayes prediction under check loss using Asymptotic Risk Estimates

Gourab Mukherjee, Lawrence D. Brown+1 more|arXiv (Cornell University)|Oct 30, 2015

Statistical Methods and Inference42 references3 citations

TL;DR

This paper develops a novel Empirical Bayes method for high-dimensional Gaussian prediction under asymmetric check loss, using asymptotically efficient risk estimates derived via Hermite polynomial expansions. The proposed shrinkage predictors are asymptotically optimal, incorporating the loss's asymmetry by targeting a pre-specified quantile of the predictive distribution rather than the mean, and outperform traditional EB methods in simulations.

ABSTRACT

We develop a novel Empirical Bayes methodology for prediction under check loss in high-dimensional Gaussian models. The check loss is a piecewise linear loss function having differential weights for measuring the amount of underestimation or overestimation. Prediction under it differs in fundamental aspects from estimation or prediction under weighted-quadratic losses. Because of the nature of this loss, our inferential target is a pre-chosen quantile of the predictive distribution rather than the mean of the predictive distribution. We develop a new method for constructing uniformly efficient asymptotic risk estimates which are then minimized to produce effective linear shrinkage predictive rules. In calculating the magnitude and direction of shrinkage, our proposed predictive rules incorporate the asymmetric nature of the loss function and are shown to be asymptotically optimal. Using numerical experiments we compare the performance of our method with traditional Empirical Bayes procedures and obtain encouraging results.

Motivation & Objective

• To address prediction in high-dimensional Gaussian models under asymmetric check loss, which penalizes underestimation and overestimation differently.
• To develop a new framework for constructing uniformly efficient asymptotic risk estimates (AREs) tailored to the piecewise linear nature of check loss.
• To produce shrinkage predictors that are asymptotically optimal by minimizing these AREs, accounting for the loss's asymmetry.
• To extend Empirical Bayes methodology beyond quadratic loss settings, particularly for quantile-based prediction.
• To validate the method’s performance through numerical experiments showing superiority over classical EB procedures.

Proposed method

• Uses a hierarchical Gaussian model with unknown hyperparameters (η, τ) for θi, where θi ∼ N(η, τ), and derives a Bayes predictor under check loss.
• Constructs asymptotically unbiased risk estimates via Hermite polynomial expansions of the relevant stochastic functions, enabling estimation of the risk function without requiring unbiasedness in the traditional quadratic sense.
• Introduces a data-driven estimator of the hyperparameter τ by minimizing the proposed asymptotically efficient risk estimate (ARE), leading to a shrinkage rule with direction and magnitude adapted to the loss asymmetry.
• Derives the shrinkage factor αi(τ) = τ / (τ + νp,i) to balance between the prior mean and observed data, with the estimator tuned to the critical ratio bi/(bi + hi) of the loss weights.
• Applies truncation and thresholding techniques (λn(i), Kn(i)) to control tail behavior in risk estimation, ensuring uniform integrability and convergence.
• Uses a grid-based optimization over τ to compute the final estimator, with theoretical guarantees of asymptotic optimality as n → ∞.

Experimental results

Research questions

• RQ1 Can asymptotically efficient risk estimates be constructed under asymmetric check loss, where traditional unbiased risk estimation fails?
• RQ2 How can Empirical Bayes prediction be adapted to target a quantile of the predictive distribution rather than the mean, under this loss?
• RQ3 What is the structure of asymptotically optimal shrinkage rules under check loss, and how do they differ from those under quadratic loss?
• RQ4 Can Hermite polynomial expansions be effectively used to estimate risk in high-dimensional models with asymmetric loss?
• RQ5 How does the performance of the proposed method compare to classical Empirical Bayes predictors in finite samples?

Key findings

• The proposed method constructs uniformly efficient asymptotic risk estimates (AREs) using Hermite polynomial expansions, enabling minimization of risk under asymmetric check loss.
• The resulting shrinkage predictors are asymptotically optimal, with the shrinkage direction and magnitude explicitly accounting for the loss’s asymmetry through the critical ratio bi/(bi + hi).
• The method outperforms traditional Empirical Bayes procedures in numerical experiments, even at moderate dimensions, demonstrating robustness and efficiency.
• Theoretical analysis confirms that the risk estimates are asymptotically unbiased and uniformly integrable under mild moment conditions.
• The estimator achieves oracle-like performance in estimating the hyperparameter τ, with convergence rates established in high-dimensional asymptotic regimes.
• The framework extends to both origin-centric and grand-mean-centric priors, with separate ARE estimators (ARE⁰, AREᴳ, AREᴰ) for each class.