[Paper Review] Efficient Estimation in Convex Single Index Models

Arun Kumar Kuchibhotla, Rohit K. Patra+1 more|arXiv (Cornell University)|Aug 1, 2017

Statistical Methods and Inference9 citations

TL;DR

This paper proposes a convex and Lipschitz constrained least squares estimator (CLSE) for single index models with an unknown convex link function, enabling efficient estimation of both parametric and nonparametric components under weak moment conditions. It establishes $n^{-1/2}$-rate asymptotic normality and semiparametric efficiency when errors have $q /ge 5$ moments and are homoscedastic, with a stable computational algorithm implemented in the R package simest.

ABSTRACT

We consider estimation and inference in a single index regression model with an unknown convex link function. We introduce a convex and Lipschitz constrained least squares estimator (CLSE) for both the parametric and the nonparametric components given independent and identically distributed observations. We prove the consistency and find the rates of convergence of the CLSE when the errors are assumed to have only $q \ge 2$ moments and are allowed to depend on the covariates. When $q\ge 5$, we establish $n^{-1/2}$-rate of convergence and asymptotic normality of the estimator of the parametric component. Moreover, the CLSE is proved to be semiparametrically efficient if the errors happen to be homoscedastic. {We develop and implement a numerically stable and computationally fast algorithm to compute our proposed estimator in the R package~ exttt{simest}}. We illustrate our methodology through extensive simulations and data analysis. Finally, our proof of efficiency is geometric and provides a general framework that can be used to prove efficiency of estimators in a wide variety of semiparametric models even when they do not satisfy the efficient score equation directly.

Motivation & Objective

• To develop a computationally stable and efficient estimator for single index models where the link function is unknown and convex.
• To establish consistency and convergence rates for the estimator under weak error moment assumptions ($q \geq 2$).
• To prove asymptotic normality and semiparametric efficiency of the parametric component estimator when errors have $q \geq 5$ moments.
• To provide a geometric framework for proving efficiency in semiparametric models beyond those satisfying the efficient score equation.
• To implement a numerically robust algorithm for practical use, available in the R package simest.

Proposed method

• Proposes a convex and Lipschitz constrained least squares estimator (CLSE) to estimate both the parametric index vector and the nonparametric convex link function.
• Imposes convexity and Lipschitz constraints on the link function to ensure regularity and uniqueness of the estimator.
• Uses empirical risk minimization with constraints to derive the CLSE, ensuring consistency under $q \geq 2$ moment conditions.
• Applies a geometric approach to prove semiparametric efficiency, bypassing the need for direct verification of the efficient score equation.
• Develops a numerically stable algorithm for computing the CLSE, implemented in the R package simest for practical deployment.
• Employs a smoothing and projection technique to handle the nonparametric component while maintaining convexity and Lipschitz properties.

Experimental results

Research questions

• RQ1 Can a convex and Lipschitz constrained least squares estimator achieve $n^{-1/2}$-rate convergence and asymptotic normality in single index models with unknown convex link functions?
• RQ2 What are the convergence rates of the CLSE when errors have only $q \geq 2$ moments and may be heteroscedastic?
• RQ3 Under what conditions is the CLSE semiparametrically efficient, particularly when errors are homoscedastic?
• RQ4 Can a geometric framework be used to prove efficiency in semiparametric models that do not satisfy the efficient score equation?
• RQ5 How can the CLSE be computed efficiently and stably in practice for real-world data analysis?

Key findings

• The CLSE achieves consistency and convergence rates under weak moment conditions, with $q \geq 2$ moments sufficient for consistency.
• When errors have $q \geq 5$ moments, the parametric component estimator attains the $n^{-1/2}$-rate of convergence and is asymptotically normal.
• The CLSE is semiparametrically efficient if the errors are homoscedastic, even without satisfying the efficient score equation directly.
• The geometric proof framework developed in the paper is generalizable to other semiparametric models beyond the current setting.
• A numerically stable and computationally efficient algorithm for the CLSE is implemented and available in the R package simest.
• Extensive simulations and data analyses confirm the estimator’s robustness and practical utility in finite samples.