[Paper Review] Detecting changes in slope with an $L_0$ penalty

Robert Maidstone, Paul Fearnhead+1 more|arXiv (Cornell University)|Jan 6, 2017

Statistical Methods and Inference21 references4 citations

TL;DR

This paper proposes CPOP, a novel dynamic programming algorithm that detects multiple changepoints in slope for univariate time series by minimizing residual sum of squares with an $L_0$ penalty on the number of changepoints. It achieves superior accuracy and parsimony over $L_1$-penalized methods and approximate alternatives like NOT and trend-filtering, particularly on real bacterial motion data with over 10,000 observations and 100+ changepoints.

ABSTRACT

Whilst there are many approaches to detecting changes in mean for a univariate time-series, the problem of detecting multiple changes in slope has comparatively been ignored. Part of the reason for this is that detecting changes in slope is much more challenging. For example, simple binary segmentation procedures do not work for this problem, whilst efficient dynamic programming methods that work well for the change in mean problem cannot be directly used for detecting changes in slope. We present a novel dynamic programming approach, CPOP, for finding the "best" continuous piecewise-linear fit to data. We define best based on a criterion that measures fit to data using the residual sum of squares, but penalises complexity based on an $L_0$ penalty on changes in slope. We show that using such a criterion is more reliable at estimating changepoint locations than approaches that penalise complexity using an $L_1$ penalty. Empirically CPOP has good computational properties, and can analyse a time-series with over 10,000 observations and over 100 changes in a few minutes. Our method is used to analyse data on the motion of bacteria, and provides fits to the data that both have substantially smaller residual sum of squares and are more parsimonious than two competing approaches.

Motivation & Objective

• To address the lack of efficient, optimal methods for detecting multiple changes in slope in time series, a problem more complex than change-in-mean detection.
• To develop a dynamic programming approach that finds the globally optimal continuous piecewise-linear fit under an $L_0$ penalty on changepoint count.
• To overcome the failure of standard binary segmentation and the inapplicability of standard dynamic programming to slope changepoint problems due to continuity constraints.
• To provide a computationally feasible, exact method for optimal segmentation that balances model fit and complexity via $L_0$ penalization.

Proposed method

• CPOP uses a continuous-state dynamic programming framework to search over all possible changepoint configurations under a piecewise-linear mean function with continuity constraints.
• The algorithm defines the optimal cost function recursively using a univariate parameter $\phi$ that captures the dependence between consecutive segments.
• It employs functional pruning to reduce computational complexity by maintaining only piecewise quadratic functions of $\phi$, enabling efficient line searches.
• The method minimizes a criterion combining residual sum of squares and an $L_0$ penalty on the number of changepoints to balance fit and model simplicity.
• It leverages the fact that the cost function is piecewise quadratic in $\phi$, allowing exact optimization via univariate minimization at each step.
• The algorithm is designed to handle large datasets (up to 10,000 observations and 100+ changepoints) with runtime in minutes.

Experimental results

Research questions

• RQ1 Can an exact dynamic programming algorithm be developed for detecting multiple changepoints in slope, despite the continuity constraint that breaks standard independence assumptions?
• RQ2 How does an $L_0$ penalty on changepoint count compare to $L_1$ penalties in terms of accuracy for estimating true changepoint locations?
• RQ3 Can functional pruning be effectively applied to piecewise quadratic cost functions in a univariate parameter space to maintain computational feasibility?
• RQ4 How does the proposed method perform on real-world data with complex, non-stationary dynamics, such as bacterial flagella motion?
• RQ5 Can CPOP outperform approximate methods like NOT and trend-filtering in both fit quality and model parsimony?

Key findings

• CPOP achieved a residual sum of squares of 1.72 on the bacterial flagella data, significantly lower than NOT’s 1.80, despite using fewer changepoints (182 vs. 794).
• For models with fewer than 350 parameters, the continuous piecewise-linear fit from CPOP had a log-likelihood 32.4 units higher than the best discontinuous mean fit.
• The CPOP segmentation had 182 changepoints and outperformed trend-filtering, which required 10,850 actual slope changes (with 278 under a 3-decimal rounding rule) and had a higher residual sum of squares (2.94).
• CPOP provided a more parsimonious and accurate fit than both NOT and trend-filtering, with substantially smaller residual sum of squares and fewer changepoints.
• The method successfully handled data with over 10,000 observations and more than 100 changepoints, completing analysis in a few minutes.
• Empirical results confirm that $L_0$ penalization leads to more reliable changepoint estimation than $L_1$ penalization in this context.