[Paper Review] Semiparametric Efficiency in Convexity Constrained Single Index Model

Arun Kumar Kuchibhotla, Rohit K. Patra+1 more|arXiv (Cornell University)|Aug 1, 2017

Statistical Methods and Inference55 references4 citations

TL;DR

This paper proposes a convex and Lipschitz-constrained least squares estimator (CLSE) for single index models with an unknown convex link function, enabling efficient estimation under weak moment conditions. It establishes $n^{-1/2}$-rate asymptotic normality and semiparametric efficiency for the parametric index when errors have $q \geq 5$ moments, using a geometric proof framework applicable beyond standard efficient score conditions.

ABSTRACT

We consider estimation and inference in a single-index regression model with an unknown convex link function. We introduce a convex and Lipschitz constrained least-square estimator (CLSE) for both the parametric and the nonparametric components given independent and identically distributed observations. We prove the consistency and find the rates of convergence of the CLSE when the errors are assumed to have only q≥2 moments and are allowed to depend on the covariates. When q≥5, we establish n−1/2-rate of convergence and asymptotic normality of the estimator of the parametric component. Moreover, the CLSE is proved to be semiparametrically efficient if the errors happen to be homoscedastic. We develop and implement a numerically stable and computationally fast algorithm to compute our proposed estimator in the R package simest. We illustrate our methodology through extensive simulations and data analysis. Finally, our proof of efficiency is geometric and provides a general framework that can be used to prove efficiency of estimators in a wide variety of semiparametric models even when they do not satisfy the efficient score equation directly. Supplementary files for this article are available online.

Motivation & Objective

• To develop a robust estimator for single index models when the link function is convex but unknown, particularly under weak error moment assumptions.
• To establish asymptotic normality and semiparametric efficiency for the parametric index component under $q \geq 5$ moments.
• To provide a numerically stable and computationally efficient algorithm for implementation via the R package simest.
• To offer a general geometric framework for proving semiparametric efficiency in models where the efficient score equation may not be directly satisfied.

Proposed method

• Proposes a convex and Lipschitz-constrained least squares estimator (CLSE) that jointly estimates the parametric index vector and the nonparametric convex link function.
• Imposes convexity and Lipschitz constraints on the link function to ensure shape-restricted estimation and stability.
• Uses an alternating minimization algorithm to compute the CLSE efficiently, with pre-binning to avoid ill-conditioning from tied or closely spaced index values.
• Applies novel maximal inequalities for unbounded and heavy-tailed errors to handle $q \geq 2$ moment assumptions.
• Employs a geometric proof strategy to establish semiparametric efficiency, avoiding reliance on the efficient score equation.
• Develops and implements a numerically stable algorithm in the R package simest for practical application.

Experimental results

Research questions

• RQ1 Can a convex and Lipschitz-constrained least squares estimator achieve $n^{-1/2}$-rate convergence and asymptotic normality for the parametric index under only $q \geq 5$ moment conditions on the errors?
• RQ2 Is the proposed CLSE semiparametrically efficient when the errors are homoscedastic, even without satisfying the standard efficient score equation?
• RQ3 How can a geometric approach to efficiency be generalized to semiparametric models where the efficient score is not directly available?
• RQ4 What are the finite-sample performance and computational feasibility of the CLSE in the presence of heteroscedastic and heavy-tailed errors?
• RQ5 Can the CLSE achieve minimax rate optimality under the given regularity conditions, particularly when $q \geq 5$?

Key findings

• The CLSE achieves $n^{-1/2}$-rate convergence and asymptotic normality for the parametric index component when the errors have $q \geq 5$ moments.
• The CLSE is semiparametrically efficient under homoscedastic errors, with the asymptotic variance equal to the Moore-Penrose inverse of the efficient information matrix.
• The proof of efficiency is geometric and generalizable, offering a new framework for proving efficiency in semiparametric models without relying on the efficient score equation.
• The CLSE is minimax rate optimal when $q \geq 5$, as shown by deriving the minimax lower bound in Section S.6.
• The proposed algorithm in the R package simest is numerically stable and computationally efficient, even with tied or closely spaced index values.
• Novel maximal inequalities are derived to handle unbounded and heavy-tailed errors, which are essential for proving convergence under $q \geq 2$ moments.