[Paper Review] Optimal Sequential Tests for Monitoring Changes in the Distribution of Finite Observation Sequences

Dong Han, Fugee Tsung+1 more|arXiv (Cornell University)|Jul 31, 2019

Advanced Statistical Process Monitoring29 references4 citations

TL;DR

This paper develops optimal sequential tests for detecting distributional changes in finite, dependent observation sequences by introducing generalized Shiryaev-type measures and dynamic control limits. It proves that different optimal tests can be constructed for various delay performance metrics and provides a formula to compute generalized out-of-control average run length, with an equivalent control limit independent of the test statistic when post-change densities are time-invariant.

ABSTRACT

This article develops a method to construct the optimal sequential test for monitoring the changes in the distribution of finite observation sequences with a general dependence structure. This method allows us to prove that different optimal sequential tests can be constructed for different performance measures of detection delay times. We also provide a formula to calculate the value of the generalized out-of-control average run length for every optimal sequential test. Moreover, we show that there is an equivalent optimal control limit which does not depend on the test statistic directly when the post-change conditional densities (probabilities) of the observation sequences do not depend on the change time.

Motivation & Objective

• To address the lack of optimal sequential tests for finite observation sequences with general dependence structures.
• To develop a framework for constructing optimal tests under different performance measures of detection delay.
• To derive a formula for computing the generalized out-of-control average run length (ARL1) for each optimal test.
• To establish an equivalent control limit that does not directly depend on the test statistic when post-change densities are time-invariant.
• To extend classical optimal tests (e.g., CUSUM, Shiryaev) to finite sequences by replacing constant limits with optimal dynamic ones.

Proposed method

• Introduces a generalized Shiryaev-type measure to evaluate detection delay performance under various metrics.
• Defines a test statistic $ Y_n $ and a dynamic control limit $ l_n(c) $ that adapt to the observation sequence length $ N $.
• Constructs optimal sequential tests $ T^*_M(c,N) $ by minimizing the generalized detection delay measure $ \mathcal{J}_{M,N}(T) $ under a fixed ARL0 constraint.
• Derives a closed-form formula to compute the generalized out-of-control ARL1 for each optimal test, ensuring minimal delay.
• Establishes an equivalent control limit that depends only on the distributional structure, not the test statistic, when post-change densities are time-homogeneous.
• Applies the framework to Markov and i.i.d. sequences, showing that classical tests remain optimal when their constant limits are replaced by dynamic equivalents.

Experimental results

Research questions

• RQ1 How can optimal sequential tests be constructed for finite observation sequences with general dependence structures?
• RQ2 What is the optimal control limit structure that ensures minimal detection delay under different performance measures?
• RQ3 Can the generalized out-of-control average run length (ARL1) be computed exactly for optimal tests in finite-horizon settings?
• RQ4 Under what conditions does an equivalent control limit exist that is independent of the test statistic?
• RQ5 How do classical tests (e.g., CUSUM, Shiryaev) perform when adapted to finite sequences using dynamic control limits?

Key findings

• Different optimal sequential tests can be constructed for different performance measures of detection delay, with each minimizing its respective generalized delay metric.
• A closed-form formula is derived to compute the generalized out-of-control average run length (ARL1) for every optimal test, ensuring minimal ARL1 under the given performance measure.
• When post-change conditional densities do not depend on the change time, an equivalent control limit exists that is independent of the test statistic, simplifying implementation.
• For i.i.d. sequences, the equivalent control limit reduces to a sequence of non-random, non-negative constants, enabling practical deployment.
• Simulation results show that the proposed optimal tests $ T^*_5 $ and $ T^*_6 $ achieve the smallest GARL5 and GARL6 values (115.43 and 23.26, respectively) under ARL0 ≈ 20, outperforming classical tests with constant limits.
• The dynamic control limits significantly improve detection performance over classical tests with fixed thresholds, especially in finite-horizon settings with Markov dependence.