[Paper Review] Element-wise estimation error of a total variation regularized estimator for change point detection

Teng Zhang|arXiv (Cornell University)|Jan 3, 2019

Statistical Methods and Inference35 references4 citations

TL;DR

This paper provides element-wise estimation error bounds for the fused lasso (total variation regularized estimator) in change point detection, enabling a thresholding-based screening method to approximately detect all change points. The analysis is non-asymptotic, holds for any regularization parameter, and achieves nearly optimal error rates, with extensions to group fused lasso.

ABSTRACT

This work studies the total variation regularized $\ell_2$ estimator (fused lasso) in the setting of a change point detection problem. Compared with existing works that focus on the sum of squared estimation errors, we give bound on the element-wise estimation error. Our bound is nearly optimal in the sense that the sum of squared error matches the best existing result, up to a logarithmic factor. This analysis of the element-wise estimation error allows a screening method that can approximately detect all the change points. We also generalize this method to the muitivariate setting, i.e., to the problem of group fused lasso.

Motivation & Objective

• To develop non-asymptotic, element-wise estimation error bounds for the fused lasso estimator in change point detection.
• To enable a screening procedure that approximately detects all change points using element-wise error control.
• To generalize the analysis to the multivariate group fused lasso setting, which is underexplored in existing literature.
• To provide error bounds that are nearly optimal, matching the best-known sum-of-squared error results up to a logarithmic factor.
• To establish theoretical guarantees that hold for any regularization parameter, avoiding fixed-parameter assumptions.

Proposed method

• Derives element-wise upper bounds on the estimation error for each observation in the fused lasso solution using geometric and vector angle analysis.
• Applies a thresholding procedure on the estimated signal to identify candidate change points based on error bounds.
• Uses vector projections and angle inequalities (e.g., sine and inverse sine bounds) to control estimation error in local signal regions.
• Employs a perturbation argument with a modified estimator to derive upper bounds on the norm of the estimated signal segment.
• Generalizes the analysis to group fused lasso by extending the geometric and error control framework to multivariate signals.
• Establishes that the error bound scales as $ O(1/ ext{polylog}( ext{signal length})) $, nearly matching the best sum-of-squared error bounds.

Experimental results

Research questions

• RQ1 Can element-wise estimation error bounds be derived for the fused lasso estimator in change point detection, independent of the regularization parameter?
• RQ2 Can such element-wise bounds be used to design a practical screening method that approximately detects all change points?
• RQ3 How does the estimation error behavior differ in the multivariate group fused lasso setting compared to the univariate case?
• RQ4 Can the error bounds be improved to match the optimal $ O(1/ ext{polylog}(n)) $ rate for the sum of squared errors?
• RQ5 Is it possible to generalize the analysis to more complex structures such as graph fused lasso or trend filtering?

Key findings

• The element-wise estimation error is bounded such that the error for each observation is controlled up to a logarithmic factor of the optimal sum-of-squared error rate.
• A thresholding-based screening method can be constructed using the element-wise error bounds to approximately detect all change points.
• The analysis holds for any regularization parameter $ heta $, making it more flexible than prior works that fix $ heta $.
• For the group fused lasso, the error bound scales as $ O(1/ ext{polylog}(n)) $, matching the univariate case up to logarithmic factors.
• The theoretical framework enables improved understanding of signal recovery in piecewise constant models with total variation regularization.
• The paper identifies open problems, including improving the $ O(1/ ext{polylog}(n)) $ bound to $ O(1/ heta) $ in the group fused lasso setting.