[Paper Review] The M-estimator in a multi-phase random nonlinear model

Gabriela Ciuperca|ArXiv.org|Jun 1, 2007

Statistical Methods and Inference16 references3 citations

TL;DR

This paper develops M-estimation for nonlinear regression models with multiple unknown change-points under random design, establishing consistency and asymptotic normality of parameter estimators. The change-point estimators converge at rate $ n^{-1} $ to the smallest minimizer of independent compound Poisson processes, valid for a broad class of error distributions, extending prior results on linear and single-change-point models.

ABSTRACT

This paper considers M-estimation of a nonlinear regression model with multiple change-points occuring at unknown times. The multi-phase random design regression model, discontinuous in each change-point, have an arbitrary error $ε$. In the case when the number of jumps is known, the M-estimator of locations of breaks and of regression parameters are studied. These estimators are consistent and the distribution of the regression parameter estimators is Gaussian. The estimator of each change-point converges, with the rate $n^{-1}$, to the smallest minimizer of the independent compound Poisson processes. The results are valid for a large class of error distributions.

Motivation & Objective

• To extend M-estimation to nonlinear, multi-phase regression models with multiple unknown change-points under random design.
• To establish the consistency and asymptotic distribution of the M-estimator for regression parameters and change-points.
• To generalize existing results on least squares and maximum likelihood estimation in multi-regime models to a robust M-estimation framework.
• To analyze the convergence rate and limiting distribution of change-point estimators under general error distributions.
• To adapt proof techniques from linear models to nonlinear settings by reworking derivative-based arguments under nonlinearity and random design.

Proposed method

• Formulates a multi-phase nonlinear regression model with $ K $ unknown change-points $ \tau_k $, where the regression function $ f_\theta(x) $ is piecewise defined using nonlinear functions $ h_{\alpha_k}(x) $.
• Defines the M-estimator by minimizing a sum of robust loss functions $ \psi(\varepsilon_i) $ over observed residuals $ Y_i - f_\theta(X_i) $, with $ \psi $ satisfying standard Huber-type conditions.
• Establishes consistency of the M-estimator $ \hat{\theta}_n = (\hat{\theta}_{1n}, \hat{\theta}_{2n}) $ for both regression parameters $ \theta_1 $ and change-points $ \theta_2 $, under regularity conditions on $ \psi $, $ h_\alpha $, and error distribution.
• Derives the asymptotic distribution of the regression parameter estimator $ \hat{\theta}_{1n} $, showing it is asymptotically Gaussian under weak moment and smoothness conditions.
• Analyzes the change-point estimator $ \hat{\theta}_{2n} $, proving $ n(\hat{\theta}_{2n} - \theta^0_2) $ converges weakly to the smallest minimizer of independent compound Poisson processes.
• Uses a local expansion of the estimating function around the true parameter, relying on derivative bounds and moment conditions on $ \partial h_\alpha / \partial \alpha $ and $ \psi $-function.

Experimental results

Research questions

• RQ1 How does M-estimation behave in a multi-phase nonlinear regression model with multiple unknown change-points under random design?
• RQ2 What are the consistency and asymptotic distribution properties of the M-estimator for both regression parameters and change-points?
• RQ3 What is the convergence rate of the change-point estimator, and what is its limiting distribution?
• RQ4 How do the results generalize prior work on least squares and maximum likelihood estimation in similar models?
• RQ5 Can the asymptotic theory for linear models be adapted to nonlinear models with random design and general error distributions?

Key findings

• The M-estimator of the regression parameters $ \hat{\theta}_{1n} $ is consistent and asymptotically Gaussian under regularity conditions.
• The change-point estimator $ \hat{\theta}_{2n} $ converges at rate $ n^{-1} $, with $ n(\hat{\theta}_{2n} - \theta^0_2) $ converging weakly to the smallest minimizer of independent compound Poisson processes.
• The asymptotic distribution of the regression parameter estimator is multivariate normal, with a limiting covariance matrix determined by the design and error distribution.
• The convergence rate $ n^{-1} $ for change-point estimation is faster than the typical $ n^{-1/2} $ rate in i.i.d. settings, due to the discontinuity at change-points.
• The results hold for a broad class of error distributions, including those with heavy tails, as long as the error density is positive and absolutely continuous.
• The proof technique adapts derivative-based expansions from linear models to nonlinear settings by controlling the behavior of $ \partial h_\alpha / \partial \alpha $ and using moment bounds on $ \psi $-functions.