[Paper Review] Quantile Processes for Semi and Nonparametric Regression

Shih-Kang Chao, Stanislav Volgushev+1 more|arXiv (Cornell University)|Apr 7, 2016

Statistical Methods and Inference31 references3 citations

TL;DR

This paper establishes weak convergence of quantile regression processes (QRP) in a general series approximation framework, covering linear models with increasing dimension, nonparametric models, and partial linear models. A key contribution is the asymptotic independence of parametric and nonparametric components in partial linear models, along with sharp Bahadur representations with exponential tail bounds for remainder terms, enabling inference under divide-and-conquer setups.

ABSTRACT

A collection of quantile curves provides a complete picture of conditional distributions. Properly centered and scaled versions of estimated curves at various quantile levels give rise to the so-called quantile regression process (QRP). In this paper, we establish weak convergence of QRP in a general series approximation framework, which includes linear models with increasing dimension, nonparametric models and partial linear models. An interesting consequence is obtained in the last class of models, where parametric and non-parametric estimators are shown to be asymptotically independent. Applications of our general process convergence results include the construction of non-crossing quantile curves and the estimation of conditional distribution functions. As a result of independent interest, we obtain a series of Bahadur representations with exponential bounds for tail probabilities of all remainder terms. Bounds of this kind are potentially useful in analyzing statistical inference procedures under divide-and-conquer setup.

Motivation & Objective

• To establish weak convergence of the quantile regression process (QRP) in a general series estimation framework for semi- and nonparametric models.
• To address the challenge of weak convergence in models with growing dimension or different convergence rates, such as partial linear models.
• To derive sharp Bahadur representations with exponential tail bounds for remainder terms, enabling robust inference under divide-and-conquer schemes.
• To demonstrate asymptotic independence between parametric and nonparametric components in partial linear models.
• To enable applications such as non-crossing quantile curve estimation and conditional distribution function estimation via weak convergence results.

Proposed method

• Develops a general framework for series estimation of conditional quantiles using basis expansions, with Z(x)⊤γn(τ) approximating the τ-th quantile Q(x; τ).
• Applies functional delta method and compact differentiability to derive weak convergence of the QRP under regularity conditions on the design, error, and basis functions.
• Establishes weak convergence of the QRP in partial linear models by proving joint weak convergence of parametric and nonparametric estimators, showing their asymptotic independence.
• Derives a fundamental Bahadur representation for series estimators with remainder terms bounded in probability using exponential tail inequalities.
• Uses covering number arguments and empirical process theory to control entropy and prove asymptotic tightness of the quantile process.
• Applies the functional delta method to maps from quantile processes to distribution functions and quantiles, enabling weak convergence of derived statistics.

Experimental results

Research questions

• RQ1 Under what conditions does the quantile regression process (QRP) weakly converge in a general series approximation framework?
• RQ2 How do the parametric and nonparametric components behave asymptotically in partial linear models, particularly in terms of joint convergence and independence?
• RQ3 Can sharp Bahadur representations with exponential tail bounds be derived for series-based quantile regression estimators?
• RQ4 What are the implications of the weak convergence results for constructing non-crossing quantile curves and estimating conditional distribution functions?
• RQ5 How do the remainder terms in Bahadur representations behave, and what are their tail properties?

Key findings

• The QRP weakly converges to a Gaussian process in the general series framework, including models with increasing dimension and nonparametric components.
• In partial linear models, the parametric and nonparametric estimators are asymptotically independent, a result derived from joint weak convergence with distinct convergence rates.
• The paper establishes a fundamental Bahadur representation with remainder terms satisfying exponential tail bounds, specifically P(|Rn| > x) ≤ C exp(−c x²) for some c, C > 0.
• The bias in series estimators is bounded as ecn = O(m−⌊η⌋/k′) for partial linear models and ecn = o(m−⌊η⌋) for univariate spline models, depending on smoothness η and dimension k′.
• Non-crossing quantile curves can be constructed via the weak convergence of the QRP, ensuring stochastic ordering across quantile levels.
• Conditional distribution functions can be consistently estimated using the functional delta method applied to the QRP, with weak convergence of the resulting process.