[Paper Review] Multiple testing under negative dependence

Ziyu Chi, Aaditya Ramdas+1 more|arXiv (Cornell University)|Dec 19, 2022

Statistical Methods in Clinical Trials4 citations

TL;DR

This paper provides the first theoretical bounds for multiple testing procedures—specifically the Simes test and Benjamini-Hochberg (BH) procedure—under various notions of negative dependence among p-values. It shows that anti-conservativeness (FDR inflation) under negative dependence is bounded by small, constant factors independent of the number of hypotheses, offering a theoretically grounded, more powerful alternative to the Benjamini-Yekutieli correction under arbitrary dependence.

ABSTRACT

The multiple testing literature has primarily dealt with three types of dependence assumptions between p-values: independence, positive regression dependence, and arbitrary dependence. In this paper, we provide what we believe are the first theoretical results under various notions of negative dependence (negative Gaussian dependence, negative regression dependence, negative association, negative orthant dependence and weak negative dependence). These include the Simes global null test and the Benjamini-Hochberg procedure, which are known experimentally to be anti-conservative under negative dependence. The anti-conservativeness of these procedures is bounded by factors smaller than that under arbitrary dependence (in particular, by factors independent of the number of hypotheses). We also provide new results about negatively dependent e-values, and provide several examples as to when negative dependence may arise. Our proofs are elementary and short, thus amenable to extensions.

Motivation & Objective

• To address the lack of theoretical understanding of multiple testing procedures under negative dependence, a common but underexplored scenario in practice.
• To establish rigorous error rate bounds for the Simes global null test and the BH procedure when p-values exhibit negative dependence.
• To demonstrate that FDR control under negative dependence incurs significantly less power loss than under arbitrary dependence, particularly compared to the Benjamini-Yekutieli correction.
• To introduce and analyze the performance of e-values under negative dependence, extending the theoretical framework to e-value-based testing.
• To identify practical scenarios where negative dependence arises and to provide a foundation for future work on adaptive and grouped multiple testing procedures.

Proposed method

• The paper introduces and formalizes five notions of negative dependence: negative Gaussian dependence, negative regression dependence, negative association, negative orthant dependence, and weak negative dependence.
• It derives bounds on the expected ratio of false discoveries to total discoveries (FDR) under the BH procedure under weak negative dependence among null p-values.
• The analysis leverages the Simes inequality and applies it under negative dependence, showing that the Simes statistic remains stochastically dominated under these conditions.
• For e-values, the paper introduces weighted merging rules and proves that e-values under negative dependence maintain valid e-value properties under specific aggregation schemes.
• Theoretical results are supported by simulations comparing BH with and without corrections under negative Gaussian dependence, using both p-values and e-values.
• The paper introduces a novel 'ND correction' for the BH procedure, which adjusts the significance level based on the strength of negative dependence, and compares it to the standard BH and Benjamini-Yekutieli corrections.

Experimental results

Research questions

• RQ1 Does the Benjamini-Hochberg procedure remain valid for FDR control under negative dependence, and if so, what is the magnitude of its anti-conservativeness?
• RQ2 Can the Simes test be theoretically justified under negative dependence, and how does its error rate compare to the independent case?
• RQ3 Do e-values under negative dependence maintain validity when merged via weighted or Simes-type rules?
• RQ4 Can the FDR inflation factor under negative dependence be bounded by a constant independent of the number of hypotheses K?
• RQ5 Is there a theoretical basis for a correction to the BH procedure under negative dependence that outperforms the Benjamini-Yekutieli correction in terms of power?

Key findings

• The FDR inflation factor for the BH procedure under weak negative dependence is bounded by a small constant, independent of the number of hypotheses K, in contrast to the log K factor under arbitrary dependence.
• The Simes test maintains its validity under negative dependence, with the expected ratio of false discoveries to discoveries bounded under weak negative dependence among null p-values.
• The proposed ND correction for the BH procedure achieves higher statistical power than the Benjamini-Yekutieli correction, especially in large-scale testing with K = 10,000 or 100,000 hypotheses.
• Simulations show that the Simes method with ND correction outperforms Bonferroni correction in both positive and negative dependence settings, with better power under negative dependence.
• The e-BH procedure and BH with ℓ_K correction are valid under arbitrary dependence but are outperformed by the BH procedure with ND correction under negative dependence, especially as K increases.
• The paper identifies that negative dependence can arise in settings such as multivariate normal data with negative correlations, and provides examples where such dependence is plausible in practice.