[Paper Review] On the mid-p-value of a test statistic with arbitrary real support

Patrick Rubin‐Delanchy, Nicholas A. Heard|arXiv (Cornell University)|May 19, 2015

Bayesian Modeling and Causal Inference28 references5 citations

TL;DR

This paper proposes the mid-p-value as a non-conservative alternative to the ordinary p-value for discrete test statistics, showing it is stochastically dominated by the uniform distribution under the convex order. The key contribution is deriving new probability bounds for functions of mid-p-values, enabling consistent inference in multiple testing scenarios where ordinary p-values fail.

ABSTRACT

The mid-p-value is a proposed improvement on the ordinary p-value for the case where the test statistic is partially or completely discrete. In this case, the ordinary p-value is conservative, meaning that its null distribution is larger than a uniform distribution on the unit interval, in the usual stochastic order. The mid-p-value is not conservative. However, as is first recognised in this article, its null distribution is dominated by the uniform distribution in a different stochastic order, called the convex order. The property leads us to discover some new probability bounds on sums, products and other functions of mid-p-values, which can be used, for example, to combine results from different hypothesis tests. Furthermore, some commonly encountered conditions are identified where combining mid-p-values, but not ordinary p-values, leads to consistent inference. Our main message is that mid-p-values need not be considered `ad-hoc'; they have some definite advantages and, under the null hypothesis, they are simply related to the uniform distribution by a different stochastic order.

Motivation & Objective

• To address the conservativeness of ordinary p-values in discrete hypothesis testing, which leads to inflated Type I error control.
• To formalize the stochastic properties of mid-p-values, particularly their relationship to the uniform distribution under the convex order.
• To derive new probability bounds for sums, products, and functions of mid-p-values to support multiple testing combination.
• To identify conditions under which combining mid-p-values leads to consistent inference, unlike ordinary p-values.

Proposed method

• The paper introduces the mid-p-value as a correction to the ordinary p-value, defined as the average of the cumulative distribution function and its left limit at the observed test statistic.
• It establishes that under the null hypothesis, the mid-p-value is stochastically dominated by the uniform distribution in the convex order, not the usual stochastic order.
• The authors derive new upper and lower bounds on the tail probabilities of sums and products of mid-p-values using convex order properties.
• They apply these bounds to construct valid combination methods for multiple hypothesis tests, particularly in settings with discrete or mixed discrete-continuous test statistics.
• Theoretical analysis is grounded in stochastic order theory, especially convex order, to characterize the distributional behavior of mid-p-values.
• The method is applied to show that mid-p-values can yield consistent inference under conditions where ordinary p-values cannot, particularly in discrete or mixed support settings.

Experimental results

Research questions

• RQ1 How does the mid-p-value compare to the ordinary p-value in terms of stochastic ordering under the null hypothesis?
• RQ2 What new probabilistic bounds can be derived for functions of mid-p-values, such as sums and products, under the convex order?
• RQ3 Under what conditions does combining mid-p-values lead to consistent inference in multiple testing scenarios?
• RQ4 Can mid-p-values be systematically used to combine results from different hypothesis tests with valid error rate control?
• RQ5 Why is the convex order a more appropriate stochastic order for analyzing mid-p-values than the usual stochastic order?

Key findings

• The mid-p-value is not conservative under the usual stochastic order, but it is dominated by the uniform distribution under the convex order.
• New probability bounds are derived for sums and products of mid-p-values, which are tighter and more informative than those for ordinary p-values.
• These bounds allow for valid combination of mid-p-values in multiple testing procedures, especially in discrete or mixed support settings.
• Under certain regularity conditions, combining mid-p-values leads to consistent inference, whereas combining ordinary p-values does not.
• The mid-p-value is not ad-hoc; it has a clear theoretical foundation rooted in convex stochastic order, making it a principled alternative to the ordinary p-value.