[Paper Review] On Combining Estimation Problems Under Quadratic Loss: A Generalization

Sévérien Nkurunziza|arXiv (Cornell University)|Sep 4, 2015

Statistical Methods and Inference20 references3 citations

TL;DR

This paper generalizes Judge and Mittelhammer's (2004) result on combining estimation problems under quadratic loss by relaxing key assumptions: it extends the dominance of Stein-type estimators to elliptically contoured distributions, removes the need for invertible variance-covariance matrices, and reduces the required dimension from k ≥ 5 to k ≥ 3 for risk dominance, regardless of correlation. The proposed class of estimators includes the Stein rule as a special case and is validated via simulation and application to the Cigarette dataset.

ABSTRACT

The main theorem in Judge and Mittelhammer [Judge, G. G., and Mittelhammer, R. (2004), A Semiparametric Basis for Combining Estimation Problems under Quadratic Loss; JASA, 99, 466, 479--487] stipulates that, in the context of nonzero correlation, a sufficient condition for the Stein rule (SR)-type estimator to dominate the base estimator is that the dimension $k$ should be at least 5. Thanks to some refined inequalities, this dominance result is proved in its full generality; for a class of estimators which includes the SR estimator as a special case. Namely, we prove that, for any member of the derived class, $k\geqslant 3$ is a sufficient condition regardless of the correlation factor. We also relax the Gaussian condition of the distribution of the base estimator, as we consider the family of elliptically contoured variates. Finally, we waive the condition on the invertibility of the variance-covariance matrix of the base and the competing estimators. Our theoretical findings are corroborated by some simulation studies, and the proposed method is applied to the Cigarette dataset.

Motivation & Objective

• To generalize Judge and Mittelhammer's (2004) result on combining estimators under quadratic loss to broader distributional assumptions.
• To remove the restrictive requirement that the variance-covariance matrix of estimators be invertible.
• To reduce the minimum dimension k required for risk dominance from 5 to 3, irrespective of correlation.
• To extend the framework to elliptically contoured distributions, broadening applicability beyond normality.
• To provide a semiparametric method for combining sample and uncertain prior information in regression models.

Proposed method

• Proposes a general class of estimators that includes the Stein rule-type estimator as a special case.
• Uses refined matrix inequalities to derive risk dominance conditions under quadratic loss.
• Relies on properties of elliptically contoured distributions to generalize results beyond normality.
• Applies a transformation to the joint distribution of base and restricted estimators to analyze risk dominance.
• Employs the Cauchy-Schwarz and Jensen’s inequalities to bound expectations involving the shrinkage function.
• Validates theoretical results through simulation studies and real-data application to the Cigarette dataset.

Experimental results

Research questions

• RQ1 Can the sufficient condition for risk dominance of Stein-type estimators be reduced from k ≥ 5 to k ≥ 3 under general dependence structures?
• RQ2 Does the risk dominance result hold when the error distribution is elliptically contoured rather than normal?
• RQ3 Can the invertibility assumption on the variance-covariance matrix of estimators be waived in the risk dominance proof?
• RQ4 Is the proposed class of estimators robust to uncertainty in prior information and applicable to low-dimensional regression models?
• RQ5 How does the proposed method perform in practice compared to existing approaches on real data?

Key findings

• The sufficient condition for risk dominance of the proposed class of estimators is k ≥ 3, regardless of correlation, improving upon the k ≥ 5 condition in Judge and Mittelhammer (2004).
• The risk dominance result holds under elliptically contoured distributions, extending applicability beyond the normal distribution.
• The invertibility assumption on the variance-covariance matrix of estimators is no longer required, enabling application in cases with linear restrictions on coefficients.
• Theoretical risk dominance is established using refined matrix inequalities and moment bounds on quadratic forms.
• Simulation studies confirm the improved risk performance of the proposed estimator under various correlation and dimension settings.
• Application to the Cigarette dataset—featuring only three regressors—demonstrates the method’s practical utility where prior methods fail.