[Paper Review] Testing Optimality of Sequential Decision-Making

Meik Dörpinghaus, Izaak Neri+3 more|arXiv (Cornell University)|Jan 4, 2018

Probabilistic and Robust Engineering Design23 references3 citations

TL;DR

This paper proposes a statistical test to determine whether a black-box sequential decision-making system is optimal, based solely on observed decision times, outcomes, and true hypotheses. It leverages fluctuation relations derived from martingale properties of likelihood ratios, showing that optimal sequential probability ratio tests exhibit decision-time distributions independent of the true hypothesis given the decision outcome—enabling a mutual information-based optimality test without requiring knowledge of observation statistics.

ABSTRACT

This paper provides a statistical method to test whether a system that performs a binary sequential hypothesis test is optimal in the sense of minimizing the average decision times while taking decisions with given reliabilities. The proposed method requires samples of the decision times, the decision outcomes, and the true hypotheses, but does not require knowledge on the statistics of the observations or the properties of the decision-making system. The method is based on fluctuation relations for decision time distributions which are proved for sequential probability ratio tests. These relations follow from the martingale property of probability ratios and hold under fairly general conditions. We illustrate these tests with numerical experiments and discuss potential applications.

Motivation & Objective

• To develop a method to test whether a sequential decision-making system is optimal in minimizing average decision time under reliability constraints.
• To address the challenge of evaluating real-world decision systems (e.g., in autonomous vehicles) where internal observation statistics are unknown or inaccessible.
• To establish a black-box testing framework that relies only on observable outputs: decision times, decisions, and true hypotheses.
• To prove that optimal sequential probability ratio tests satisfy specific symmetry conditions in decision-time distributions across hypotheses.
• To enable practical validation of decision algorithms without access to underlying signal processing models or observation processes.

Proposed method

• The method tests optimality by evaluating whether the decision-time distribution is conditionally independent of the true hypothesis given the decision outcome.
• It uses fluctuation relations derived from the martingale property of likelihood ratios in sequential probability ratio tests (SPRTs), valid under general i.i.d. observation conditions.
• The core test checks if the conditional probability density of decision time p_T(t|H, D) is identical for both hypotheses H=1 and H=2 when the decision outcome D is fixed.
• A mutual information measure I(H; T) between the true hypothesis H and decision time T is computed; optimality implies I(H; T) = 0 under the derived symmetry conditions.
• The method is non-parametric and does not require knowledge of the observation process or the decision algorithm’s internal structure.
• Numerical experiments validate the test’s ability to detect deviations from optimality using empirical decision-time distributions.

Experimental results

Research questions

• RQ1 Can we determine whether a black-box sequential decision system is optimal without access to its internal observation statistics or algorithmic design?
• RQ2 What statistical conditions must the decision-time distribution satisfy for a sequential test to be optimal in minimizing mean decision time under fixed error rates?
• RQ3 Does the decision time distribution in an optimal sequential test carry any information about the true hypothesis beyond the decision outcome?
• RQ4 How can fluctuation relations for likelihood ratios be used to derive testable necessary conditions for optimality in sequential analysis?
• RQ5 Can mutual information between hypothesis and decision time serve as a quantitative metric for optimality in sequential decision-making?

Key findings

• Optimal sequential probability ratio tests satisfy the symmetry condition: p_T(t|H=1, D=d) = p_T(t|H=2, D=d) for both decision outcomes d ∈ {1, 2}.
• This symmetry implies that the decision time T contains no additional information about the true hypothesis H beyond the decision outcome D, leading to zero mutual information I(H; T) = 0 under optimality.
• The mutual information I(H; T) is proven to be zero under the derived fluctuation relations, which stem from the martingale property of likelihood ratios.
• The proposed test is valid under general i.i.d. observation processes and does not require knowledge of the observation statistics or decision algorithm.
• Numerical experiments confirm that the test correctly identifies optimal systems and detects suboptimal behavior in simulated decision-making processes.
• The method enables practical validation of real-world systems such as autonomous vehicle perception modules using only observable decision outputs.