[Paper Review] Asymptotic Equivalence for Nonparametric Regression

Ion Grama, Michael Nussbaum|arXiv (Cornell University)|Dec 19, 2024

Statistical Methods and Inference15 references38 citations

TL;DR

The paper shows that under regularity assumptions, a nonparametric regression model with non-Gaussian noise is asymptotically equivalent to a Gaussian nonparametric regression model via a variance-stabilizing transformation Gamma() function related to Fisher information.

ABSTRACT

We consider a nonparametric model $\mathcal{E}^{n},$ generated by independent observations $X_{i},$ $i=1,...,n,$ with densities $p(x,θ_{i}),$ $i=1,...,n,$ the parameters of which $θ_{i}=f(i/n)\in Θ$ are driven by the values of an unknown function $f:[0,1] ightarrow Θ$ in a smoothness class. The main result of the paper is that, under regularity assumptions, this model can be approximated, in the sense of the Le Cam deficiency pseudodistance, by a nonparametric Gaussian shift model $Y_{i}=Γ(f(i/n))+\varepsilon _{i},$ where $\varepsilon_{1},...,\varepsilon _{n}$ are i.i.d. standard normal r.v.'s, the function $Γ(θ):Θ ightarrow \mathrm{R}$ satisfies $Γ^{\prime}(θ)=\sqrt{I(θ)}$ and $I(θ)$ is the Fisher information corresponding to the density $p(x,θ).$

Motivation & Objective

• Motivate the study of asymptotic equivalence between nonparametric regression models with non-Gaussian noise and Gaussian shift models.
• Introduce a general regularity framework for parametric families and derive sufficient conditions for asymptotic equivalence.
• Show a local-to-global globalization strategy to extend local Gaussian approximations to the full nonparametric model.
• Relate the variance-stabilizing transformation  and the function  to the Fisher information in the parametric family.
• Provide concrete examples illustrating the scope of the theory, including location models and exponential family models.

Proposed method

• Formulate a nonparametric model driven by a smooth function f in a Hölder ball and a parametric density p(x,).
• Establish a local Gaussian approximation: Y_i^n = h(i/n) + I(f(i/n))^{-1/2} _i with i.i.d. normal _i.
• Prove a global asymptotic equivalence to a Gaussian shift model with Y_i = Gamma(f(i/n)) + _i, where Gamma'()=sqrt(I()).
• Use a local-to-global globalization scheme combining block-wise approximations and a Hungarian construction for likelihood processes.
• Develop a general local nonparametric theory (LASE) giving Hellinger distance control between nonparametric and Gaussian experiments.
• Apply regularity assumptions (R1-R3) and growth conditions (G1-G2) to ensure the asymptotic equivalence.

Experimental results

Research questions

• RQ1 Can a nonparametric regression model with non-Gaussian noise be approximated by a Gaussian shift model as sample size grows?
• RQ2 What regularity conditions on the density p(x,) and the Fisher information guarantee asymptotic equivalence?
• RQ3 How does the variance-stabilizing transformation Gamma relate to Fisher information in this equivalence?
• RQ4 Can local Gaussian approximations be globalized over time and function spaces to yield a global equivalence?
• RQ5 What are concrete examples (e.g., location models, exponential family models) that illustrate the theory?

Key findings

• Under beta > 1/2, the nonparametric model with densities p(x,f(t)) is asymptotically equivalent to a Gaussian shift model with Y_i = Gamma(f(i/n)) + epsilon_i.
• Gamma'() = sqrt(I()) where I() is the Fisher information of p(x,).
• The local Gaussian approximation holds uniformly for f in the Hölder class Sigma^beta, and can be globalized to the full nonparametric class.
• In the location model, Gamma( heta) = , recovering a simple Gaussian regression with noise scaled by Fisher information.
• The exponential family and Bernoulli, Gaussian scale, and Poisson examples illustrate the range of asymptotic equivalence through explicit Gamma mappings.
• The theory explains how variance-stabilizing transformations arise naturally in establishing equivalent Gaussian models.