[Paper Review] Asymptotics for penalized spline estimators in quantile regression

Takuma Yoshida|arXiv (Cornell University)|Sep 6, 2012

Statistical Methods and Inference8 references3 citations

TL;DR

This paper establishes the asymptotic normality of penalized spline estimators in quantile regression using a low-rank B-spline model with difference penalties. It derives the asymptotic distribution by analyzing model bias and penalty-induced bias, showing the estimator converges to a normal distribution under regularity conditions, with convergence rate $ O(n^{-(p+1)/(2p+3)}) $ for the bias term.

ABSTRACT

Quantile regression predicts the $τ$-quantile of the conditional distribution of a response variable given the explanatory variable for $τ\in(0,1)$. The aim of this paper is to establish the asymptotic distribution of the quantile estimator obtained by penalized spline method. A simulation and an exploration of real data are performed to validate our results.

Motivation & Objective

• To establish the asymptotic distribution of penalized spline estimators in quantile regression, addressing a gap in theoretical understanding.
• To analyze and quantify the two sources of bias: model bias from B-spline approximation and penalty-induced bias from the difference penalty.
• To derive the asymptotic normality of the penalized spline quantile estimator under regularity conditions on the design and error structure.
• To validate the theoretical findings through simulation and real data analysis, demonstrating practical applicability.
• To provide a computationally efficient alternative to smoothing splines with comparable smoothness and robustness to outliers in quantile regression.

Proposed method

• Uses a low-rank B-spline basis expansion to approximate the conditional quantile function $ \eta_\tau(x) $, with $ p $-th degree B-spline basis functions $ B_k^{[p]}(x) $.
• Defines the penalized spline estimator as the minimizer of a convex loss function combining the check function $ \rho_\tau $ and a difference penalty term involving the second-order difference matrix $ D_m $.
• Applies the penalized iteratively reweighted least squares (PIRLS) method to compute the estimator efficiently.
• Derives the asymptotic distribution by analyzing the joint behavior of the stochastic term $ U_{1n}(\boldsymbol{\delta}) $, the empirical process term $ U_{2n}(\boldsymbol{\delta}) $, and the penalty term $ U_{3n}(\boldsymbol{\delta}) $.
• Establishes asymptotic normality by showing the limiting distribution of $ \sqrt{n/K_n} \{ \hat{\mathbf{b}}(\tau) - \mathbf{b}^*(\tau) \} $ converges to a normal distribution with mean zero and covariance depending on $ G(\tau) $ and the penalty matrix.
• Uses the properties of B-spline derivatives and the asymptotic behavior of the coefficient vector $ \mathbf{b}^*(\tau) $ to derive the order of magnitude of the bias term $ b^\lambda_\tau(x) $.

Experimental results

Research questions

• RQ1 What is the asymptotic distribution of the penalized spline estimator in quantile regression under a B-spline basis with difference penalty?
• RQ2 How do the model bias (from B-spline approximation) and penalty-induced bias contribute to the overall asymptotic bias of the estimator?
• RQ3 What is the convergence rate of the bias term $ b^\lambda_\tau(x) $, and how does it depend on the sample size $ n $, the number of knots $ K_n $, and the spline degree $ p $?
• RQ4 Under what conditions does the penalized spline quantile estimator achieve asymptotic normality?
• RQ5 How does the proposed method compare in finite samples to existing nonparametric quantile regression methods in terms of bias and variance?

Key findings

• The penalized spline quantile estimator is asymptotically normal, with $ \sqrt{n/K_n} \{ \hat{\eta}_\tau(x) - \eta^*_\tau(x) - b^\lambda_\tau(x) \} \stackrel{D}{\to} N(0, \Phi_\tau(x)) $, where $ \Phi_\tau(x) = O(1) $.
• The asymptotic bias $ b^\lambda_\tau(x) $ from the penalty term is of order $ O(n^{-(p+1)/(2p+3)}) $, which depends on the spline degree $ p $ and the sample size $ n $.
• The convergence rate of the estimator is $ O(n^{-(p+1)/(2p+3)}) $, matching the optimal rate for nonparametric quantile regression under regularity conditions.
• The bias due to the B-spline approximation is of order $ O(K_n^{-(p+1)}) $, and the penalty-induced bias is of order $ O(\lambda_n n^{-1} K_n^{1-m}) $, with $ m = 2 $ for second-order differences.
• The asymptotic variance of the estimator is proportional to $ \tau(1-\tau) $, consistent with the known variance of the check function in quantile regression.
• The theoretical results are supported by simulation and real data analysis, confirming the validity and practicality of the proposed method.