[Paper Review] Piecewise Convex Function Estimation and Model Selection

Kurt S. Riedel|arXiv (Cornell University)|Mar 11, 2018

Statistical Methods and Inference4 references3 citations

TL;DR

This paper proposes a two-stage method for estimating piecewise convex functions from noisy data, where the function is assumed to have a small number of convexity change points. It uses a pilot estimator to detect inflection points and constructs uncertainty intervals to guide model selection, ensuring geometric fidelity by constraining the fit to match the true convexity structure with asymptotically optimal smoothing.

ABSTRACT

Given noisy data, function estimation is considered when the unknown function is known apriori to consist of a small number of regions where the function is either convex or concave. When the regions are known apriori, the estimate is reduced to a finite dimensional convex optimization in the dual space. When the number of regions is unknown, the model selection problem is to determine the number of convexity change points. We use a pilot estimator based on the expected number of false inflection points.

Motivation & Objective

• To estimate functions that are piecewise convex or concave with a small number of convexity change points from noisy data.
• To address the model selection problem of determining the number and location of convexity change points when they are a priori unknown.
• To preserve geometric fidelity in the estimate by aligning the number and location of convexity changes in the fitted function with those of the true function.
• To develop a data-driven method that selects the correct convexity structure with high probability, minimizing model misspecification error.
• To achieve asymptotically optimal convergence rates by using generalized cross-validation for smoothing parameter selection in a constrained regression framework.

Proposed method

• Uses a pilot estimator based on unconstrained smoothing to estimate the ℓ-th and (ℓ+1)-th derivatives of the function.
• Constructs uncertainty intervals around estimated inflection points using asymptotic normality, with width proportional to σ²‖κ⁽ℓ⁾‖²F′(s)/|f^(ℓ+1)(x̂j)|²nh²ℓ+1.
• Applies a two-stage estimation: first, detects candidate change points via pilot estimates; second, performs constrained regression in a closed convex cone matching the detected structure.
• Imposes constraints on f^(ℓ) or f^(ℓ+1) to be non-negative or non-positive in each uncertainty interval, depending on the parity of overlapping intervals.
• Uses a penalized functional VP[f] = (λ/p)∫|f^(m)|^p ds + ∑ψi(⟨hi,f⟩−yi) to enforce smoothness and fidelity, with p=2 for least squares.
• Selects smoothing parameters via generalized cross-validation (GCV), with λn = λ_GCV and h_n = ι(n)h_GCV, where ι(n) = log²(n)n^(1/(2ℓ+3)−1/(2m+1)).

Experimental results

Research questions

• RQ1 How can we estimate a function with a small number of convexity change points when the number and location of these points are unknown?
• RQ2 What is the optimal way to select the number and location of convexity change points to preserve geometric fidelity in the estimate?
• RQ3 How can we construct uncertainty intervals around estimated inflection points to guide model selection in a statistically valid way?
• RQ4 What conditions ensure that the constrained estimator achieves asymptotically optimal convergence rates?
• RQ5 How can we avoid oversmoothing near true inflection points while still eliminating spurious ones?

Key findings

• The two-stage estimator with uncertainty interval-based model selection achieves high probability of correctly identifying the true convexity structure, with P(correct model) → 1 as n → ∞.
• The asymptotic mean squared error of the estimator satisfies E‖f̂−f‖²_j ∼ α_jλ^(m−j)/m‖f‖_m² + β_jσ²/(nλ^(2k+1)/(2m)), under appropriate smoothing parameter scaling.
• The method avoids oversmoothing by using asymptotically optimal smoothing parameters, unlike earlier schemes that minimize the number of inflection points post-hoc.
• The convergence is uniform on compact subintervals [δ,1−δ], and the solution is in C^{2m−ℓ−2} and satisfies the Euler-Lagrange equation (2.4) in regions where |f^(ℓ)| > 0.
• The error bound holds under the conditions liminf λ_n^(1/2m)n^(1/(2ℓ+3)) > 0 and limsup < ∞, with ℓ < 2m−5/2.
• The use of uncertainty intervals to guide constraint selection ensures that the number of false inflection points is controlled, with expected number of false positives vanishing as n increases.