[Paper Review] Robust and Scalable Bayesian Online Changepoint Detection

Matias Altamirano, François‐Xavier Briol+1 more|arXiv (Cornell University)|Feb 9, 2023

Statistical Methods and Inference4 citations

TL;DR

This paper introduces ${\mathcal{D}}_{m}$-BOCD, a robust and scalable Bayesian online changepoint detection algorithm that leverages diffusion score matching to achieve provably robust inference and closed-form posterior updates. It is over 10× faster than its closest competitor ($\beta$-BOCD) while maintaining accuracy on real-world data with outliers, such as the 2013 Twitter flash crash and UK government bond yields.

ABSTRACT

This paper proposes an online, provably robust, and scalable Bayesian approach for changepoint detection. The resulting algorithm has key advantages over previous work: it provides provable robustness by leveraging the generalised Bayesian perspective, and also addresses the scalability issues of previous attempts. Specifically, the proposed generalised Bayesian formalism leads to conjugate posteriors whose parameters are available in closed form by leveraging diffusion score matching. The resulting algorithm is exact, can be updated through simple algebra, and is more than 10 times faster than its closest competitor.

Motivation & Objective

• To address the lack of robustness in standard Bayesian online changepoint detection (BOCD) under model misspecification and outliers.
• To overcome the scalability limitations of existing robust methods like $\beta$-BOCD, which rely on expensive variational approximations.
• To develop a generalised Bayesian framework that ensures robustness while enabling exact, closed-form posterior updates for exponential family models.
• To demonstrate that the proposed method is computationally efficient—matching standard BOCD in complexity—while outperforming $\beta$-BOCD in speed and robustness.
• To extend the applicability of Bayesian changepoint detection to multivariate, non-Gaussian, and intractable-likelihood settings where standard conjugate priors fail.

Proposed method

• Proposes a new generalised posterior, ${\mathcal{D}}_{m}$-posterior, derived via diffusion score matching, a weighted variant of Hyvärinen's score matching.
• Uses the diffusion matrix $m$ to tune robustness, ensuring the posterior is robust to outliers and model misspecification.
• Establishes that for exponential family likelihoods with normal or truncated normal priors, ${\mathcal{D}}_{m}$-posteriors are conjugate and update via elementary algebraic operations.
• Integrates the ${\mathcal{D}}_{m}$-posterior into the Bayesian online changepoint detection (BOCD) framework, enabling efficient online inference.
• Employs pruning strategies from Adams & MacKay (2007) to maintain computational efficiency, resulting in $\mathcal{O}(T(d^2 + p^2))$ complexity.
• Applies the method to real-world data, including financial time series and geological data, with and without outliers, to validate robustness and speed.

Experimental results

Research questions

• RQ1 Can a generalised Bayesian approach based on diffusion score matching provide provable robustness in online changepoint detection?
• RQ2 Can such a method achieve closed-form posterior updates for exponential family models, enabling efficient online computation?
• RQ3 Does the resulting algorithm scale effectively compared to existing robust methods like $\beta$-BOCD?
• RQ4 Can the method detect true changepoints accurately in the presence of outliers, where standard BOCD fails?
• RQ5 In settings where standard Bayesian posteriors are intractable (e.g., mixed exponential and Gaussian components), does ${\mathcal{D}}_{m}$-BOCD remain computationally feasible and accurate?

Key findings

• On the 2013 Twitter flash crash data, standard BOCD falsely detected three changepoints, while ${\mathcal{D}}_{m}$-BOCD correctly detected none, demonstrating robustness.
• $\mathcal{D}$-m-BOCD is over 10 times faster than $\beta$-BOCD in numerical experiments, with comparable or better detection accuracy.
• For multivariate data with mixed exponential and Gaussian components, ${\mathcal{D}}_{m}$-BOCD is computationally feasible, while standard BOCD requires expensive sampling or variational approximations.
• On UK 10-year government bond yields modeled with a gamma likelihood, ${\mathcal{D}}_{m}$-BOCD produces a tractable, truncated Gaussian posterior, whereas standard conjugate priors lead to intractable normalisation constants.
• The algorithm maintains the same computational complexity as standard BOCD ($\mathcal{O}(T(d^2 + p^2))$), making it scalable for high-frequency data.
• The method successfully detects meaningful changepoints in the bond yield data that align with major political events, confirming its practical utility.