[Paper Review] An Efficient ADMM Algorithm for Structural Break Detection in Multivariate Time Series

Alex Tank, Emily B. Fox+1 more|arXiv (Cornell University)|Nov 22, 2017

Blind Source Separation Techniques6 references3 citations

TL;DR

This paper proposes an efficient ADMM algorithm for detecting structural breaks in multivariate time series by formulating a convex optimization problem with group fused lasso penalties. By reparameterizing the problem and leveraging Kalman smoothing, the method solves the global subproblem in linear time per dimension, enabling exact, scalable detection of change points in vector autoregressive models with theoretical guarantees and computational efficiency.

ABSTRACT

We present an efficient alternating direction method of multipliers (ADMM) algorithm for segmenting a multivariate non-stationary time series with structural breaks into stationary regions. We draw from recent work where the series is assumed to follow a vector autoregressive model within segments and a convex estimation procedure may be formulated using group fused lasso penalties. Our ADMM approach first splits the convex problem into a global quadratic program and a simple group lasso proximal update. We show that the global problem may be parallelized over rows of the time dependent transition matrices and furthermore that each subproblem may be rewritten in a form identical to the log-likelihood of a Gaussian state space model. Consequently, we develop a Kalman smoothing algorithm to solve the global update in time linear in the length of the series.

Motivation & Objective

• To develop an exact, scalable algorithm for detecting structural breaks in non-stationary multivariate time series.
• To overcome limitations of approximate solvers like group least angle regression, which lack optimality guarantees.
• To exploit the time series structure to enable parallelization and linear-time computation via Kalman smoothing.
• To provide a convex optimization framework that jointly estimates breakpoints and autoregressive parameters with group fused lasso penalties.
• To ensure computational efficiency and scalability for moderate to high-dimensional multivariate processes.

Proposed method

• Reformulate the structural break detection problem as a convex optimization with group fused lasso penalties on time-varying VAR parameters.
• Introduce a change of variables to decouple the least squares and group lasso terms, enabling ADMM decomposition.
• Apply ADMM to split the problem into a global quadratic subproblem (solved via Kalman smoothing) and a proximal group lasso update.
• Rewrite the global subproblem as a Gaussian state space model, allowing the use of the Rauch-Tung-Streibel smoother for efficient solution.
• Perform forward filtering and backward smoothing steps in linear time per dimension, with each step involving matrix inversions of size $pK \times pK$.
• Update the $W$ variable via a group soft-thresholding operation, ensuring sparsity in the difference of consecutive parameter matrices.

Experimental results

Research questions

• RQ1 Can an exact optimization algorithm be developed for structural break detection in multivariate time series that guarantees convergence to the global optimum?
• RQ2 How can the computational complexity of solving the global subproblem be reduced while preserving statistical accuracy?
• RQ3 To what extent can the time series structure be exploited to parallelize and accelerate the solution of the convex optimization problem?
• RQ4 Can Kalman smoothing be effectively adapted to solve the global update in ADMM for group fused lasso problems in time series segmentation?
• RQ5 How does the proposed ADMM-based method compare in accuracy and efficiency to approximate solvers like group least angle regression?

Key findings

• The proposed ADMM algorithm solves the global subproblem in time linear in the series length $N$, achieving computational efficiency.
• The Kalman smoothing approach enables exact solution of the global quadratic subproblem by mapping it to a Gaussian state space model.
• The method achieves exact optimization with theoretical convergence guarantees, unlike approximate solvers such as group least angle regression.
• In simulations with $p=10$, $N=300$, and two true break points at $t=100$ and $t=200$, the algorithm correctly detected breakpoints with $\lambda=5$.
• The algorithm scales well for moderate $p$, with each recursive smoothing step requiring only $O(pK)$ matrix operations per time point.
• The backward smoothing phase involves $pK \times pK$ matrix inversions, which is feasible for moderate dimensions but a bottleneck in high-dimensional settings.