[Paper Review] Minimax Risk Bounds for Piecewise Constant Models

Chao Gao, Fang Han+1 more|arXiv (Cornell University)|May 18, 2017

Statistical Methods and Inference11 citations

TL;DR

This paper establishes the exact minimax rate for estimating piecewise constant and monotone sequences, revealing an iterated logarithmic dependence on dimension due to isotonic constraints. It proposes a penalized least-squares estimator that adapts to the unknown structure, achieves the optimal rate, and is computationally efficient even under model misspecification.

ABSTRACT

Consider a sequence of real data points $X_1,\ldots, X_n$ with underlying means $ heta^*_1,\dots, heta^*_n$. This paper starts from studying the setting that $ heta^*_i$ is both piecewise constant and monotone as a function of the index $i$. For this, we establish the exact minimax rate of estimating such monotone functions, and thus give a non-trivial answer to an open problem in the shape-constrained analysis literature. The minimax rate involves an interesting iterated logarithmic dependence on the dimension, a phenomenon that is revealed through characterizing the interplay between the isotonic shape constraint and model selection complexity. We then develop a penalized least-squares procedure for estimating the vector $ heta^*=( heta^*_1,\dots, heta^*_n)^T$. This estimator is shown to achieve the derived minimax rate adaptively. For the proposed estimator, we further allow the model to be misspecified and derive oracle inequalities with the optimal rates, and show there exists a computationally efficient algorithm to compute the exact solution.

Motivation & Objective

• To resolve an open problem in shape-constrained estimation by determining the exact minimax rate for monotone, piecewise constant functions.
• To understand the interplay between isotonic shape constraints and model selection complexity in high-dimensional settings.
• To develop a penalized least-squares estimator that adaptively achieves the minimax rate without prior knowledge of the underlying structure.
• To extend the estimator’s theoretical guarantees to misspecified models, providing optimal oracle inequalities.
• To design a computationally efficient algorithm that computes the exact solution of the estimator.

Proposed method

• The paper analyzes the minimax risk over the class of sequences that are both piecewise constant and monotone in index.
• It derives the minimax rate by characterizing the complexity of the shape-constrained model class, revealing an iterated logarithmic dependence on the sample size n.
• A penalized least-squares estimator is proposed, where the penalty term balances fit and the number of jumps in the piecewise constant function.
• The estimator is shown to achieve the derived minimax rate adaptively, without requiring knowledge of the true number of change points.
• Oracle inequalities are derived under model misspecification, showing the estimator maintains optimal convergence rates.
• A computationally efficient algorithm is constructed to compute the exact solution of the penalized estimator, leveraging dynamic programming or active set methods.

Experimental results

Research questions

• RQ1 What is the exact minimax rate of estimation for sequences that are both piecewise constant and monotone in index?
• RQ2 How does the isotonic shape constraint affect the model selection complexity and the resulting minimax risk?
• RQ3 Can a penalized least-squares estimator achieve the minimax rate adaptively across different unknown configurations of the true sequence?
• RQ4 What are the theoretical guarantees of the estimator when the model is misspecified?
• RQ5 Is there a computationally efficient algorithm to compute the exact solution of the proposed estimator?

Key findings

• The minimax rate for estimating monotone, piecewise constant sequences involves an iterated logarithmic factor in the dimension, reflecting the complex interplay between shape constraints and model complexity.
• The proposed penalized least-squares estimator achieves the derived minimax rate adaptively, without requiring prior knowledge of the number of change points.
• Under model misspecification, the estimator satisfies oracle inequalities with optimal rates, demonstrating robustness to deviations from the assumed model.
• An exact solution to the estimator can be computed efficiently, enabling practical implementation despite the non-convex nature of the shape constraint.
• The minimax rate is strictly faster than the standard piecewise constant case due to the additional monotonicity constraint, which reduces the effective model complexity.