[Paper Review] Differentially Private Change-Point Detection

Rachel Cummings, Sara Krehbiel+3 more|arXiv (Cornell University)|Aug 29, 2018

Privacy-Preserving Technologies in Data16 citations

TL;DR

This paper presents the first differentially private algorithms for both offline and online change-point detection, leveraging ReportMax and AboveThresh mechanisms to ensure $\epsilon$-differential privacy. It provides finite-sample accuracy guarantees for the maximum likelihood estimator under privacy constraints, quantifying the privacy cost as $D_{KL}(P_0||P_1)/\epsilon$, with empirical validation showing strong performance in detecting change-points under varying privacy levels.

ABSTRACT

The change-point detection problem seeks to identify distributional changes at an unknown change-point k* in a stream of data. This problem appears in many important practical settings involving personal data, including biosurveillance, fault detection, finance, signal detection, and security systems. The field of differential privacy offers data analysis tools that provide powerful worst-case privacy guarantees. We study the statistical problem of change-point detection through the lens of differential privacy. We give private algorithms for both online and offline change-point detection, analyze these algorithms theoretically, and provide empirical validation of our results.

Motivation & Objective

• To address the gap in differentially private change-point detection for sensitive data streams in applications like biosurveillance and IoT.
• To provide the first finite-sample accuracy bounds for the non-private maximum likelihood estimator (MLE) of the change-point.
• To design and analyze private algorithms that ensure $\epsilon$-differential privacy while maintaining statistical accuracy.
• To extend offline private detection to the online setting using the AboveThresh mechanism with sliding windows.
• To empirically validate the performance of private change-point detection under varying privacy parameters $\epsilon$ and distributional assumptions.

Proposed method

• Uses the ReportMax mechanism to privately estimate the MLE of the change-point in the offline setting by adding Laplace noise proportional to the sensitivity of partial log-likelihood queries.
• Applies the AboveThresh mechanism in the online setting to detect a window of size $n$ containing the change-point by iteratively querying noisy statistics until a threshold is exceeded.
• Employs a recursive computation of CUSUM statistics: $W_i = \max(W_{i-1}, 0) + \log(P_1(x_i)/P_0(x_i))$ to efficiently compute the test statistic over sliding windows.
• Sets the detection threshold $T$ using empirical quantiles of noisy CUSUM statistics under pre-change and post-change distributions to control false alarm rates.
• Uses Monte Carlo simulations with $10^6$ runs to estimate false alarm and detection accuracy, with threshold selection constrained to ensure non-empty range for reliable detection.
• Applies sensitivity-based noise injection with $\Delta/\epsilon$ scaling to achieve $\epsilon$-differential privacy, where $\Delta$ is the sensitivity of the log-likelihood statistic.

Experimental results

Research questions

• RQ1 What is the finite-sample accuracy of the non-private MLE for change-point detection, and how does it compare to private estimators?
• RQ2 How can the ReportMax mechanism be adapted to privately estimate the change-point in an offline data stream?
• RQ3 Can the AboveThresh mechanism be used to enable private change-point detection in an online setting without prior knowledge of the change-point location?
• RQ4 What is the trade-off between privacy ($\epsilon$) and detection accuracy in both offline and online settings?
• RQ5 How do window size and distributional assumptions affect the threshold selection and detection performance in online private change-point detection?

Key findings

• The paper provides the first finite-sample accuracy guarantees for the non-private MLE of the change-point, showing that accuracy is independent of database size $n$ under bounded sensitivity.
• Private offline detection via ReportMax achieves $\epsilon$-differential privacy and accuracy bounded by $D_{KL}(P_0||P_1)/\epsilon$, quantifying the cost of privacy.
• In online simulations with $n=700$ and $k^*=5000$, the algorithm achieves a false alarm rate below 0.10 for $\epsilon=\infty$ and $\epsilon=1$ in the Bernoulli model, with $T=220$.
• For the Gaussian model, a larger window size is needed; with $n=700$, a relaxed false alarm constraint was required for $\epsilon=0.5,1$, using $T=180,150,100$ respectively.
• Conditional detection accuracy in the online setting is comparable to offline accuracy, indicating that the main challenge lies in alarm triggering, not estimation precision.
• Empirical results show that once a correct window is identified via AboveThresh, the offline estimation algorithm accurately locates the change-point, confirming the effectiveness of the modular design.