[Paper Review] A New Approach for Large Scale Multiple Testing with Application to FDR Control for Graphically Structured Hypotheses

Wenge Guo, Gavin Lynch+1 more|arXiv (Cornell University)|Dec 1, 2018

Statistical Methods in Clinical Trials27 references5 citations

TL;DR

This paper introduces a novel general framework for large-scale multiple testing that enables the development of false discovery rate (FDR)-controlling procedures for hypotheses with a directed acyclic graph (DAG) structure. By transforming the FDR control problem into a per-family error rate (PFER) control problem, the method ensures rejected hypotheses preserve the original DAG structure, improving power and interpretability compared to standard procedures like BH.

ABSTRACT

In many large scale multiple testing applications, the hypotheses often have a known graphical structure, such as gene ontology in gene expression data. Exploiting this graphical structure in multiple testing procedures can improve power as well as aid in interpretation. However, incorporating the structure into large scale testing procedures and proving that an error rate, such as the false discovery rate (FDR), is controlled can be challenging. In this paper, we introduce a new general approach for large scale multiple testing, which can aid in developing new procedures under various settings with proven control of desired error rates. This approach is particularly useful for developing FDR controlling procedures, which is simplified as the problem of developing per-family error rate (PFER) controlling procedures. Specifically, for testing hypotheses with a directed acyclic graph (DAG) structure, by using the general approach, under the assumption of independence, we first develop a specific PFER controlling procedure and based on this procedure, then develop a new FDR controlling procedure, which can preserve the desired DAG structure among the rejected hypotheses. Through a small simulation study and a real data analysis, we illustrate nice performance of the proposed FDR controlling procedure for DAG-structured hypotheses.

Motivation & Objective

• To address the challenge of controlling the false discovery rate (FDR) in large-scale multiple testing when hypotheses have a known graphical (DAG) structure.
• To develop a general methodological framework that simplifies the construction of FDR-controlling procedures by reducing it to the design of PFER-controlling procedures.
• To ensure that the set of rejected hypotheses maintains the original DAG structure, preserving hierarchical integrity for improved interpretability.
• To demonstrate improved statistical power over standard procedures like BH while maintaining strong error rate control under independence.
• To provide theoretical guarantees for FDR control under the proposed framework, particularly for DAG-structured hypotheses.

Proposed method

• Propose a general approach (Theorem 3.1) that reduces FDR control to PFER control, simplifying the design of new multiple testing procedures.
• Develop a specific PFER-controlling procedure for DAG-structured hypotheses under the assumption of independence (Theorem 4.1), using a recursive weighting scheme based on parent-child relationships.
• Construct a new FDR-controlling procedure (Theorem 5.1) by transforming the PFER-controlling procedure, ensuring that a hypothesis is rejected only if all its parent hypotheses are rejected.
• Use a critical value function involving the inverse survival function of a compound Poisson distribution to calibrate rejection thresholds while preserving the DAG structure.
• Apply a recursive algorithm that processes hypotheses in topological order, starting from leaves and moving toward roots, to compute critical values and rejection decisions.
• Leverage the fact that the PFER control of the derived procedure implies FDR control under independence, as shown via coupling arguments and indicator variable bounds.

Experimental results

Research questions

• RQ1 Can a general framework be developed to simplify the construction of FDR-controlling procedures for structured hypotheses?
• RQ2 How can the per-family error rate (PFER) control framework be leveraged to derive valid FDR-controlling procedures for DAG-structured hypotheses?
• RQ3 To what extent does incorporating DAG structure improve statistical power compared to standard FDR procedures like BH?
• RQ4 Can the proposed method ensure that the set of rejected hypotheses forms a valid sub-DAG, preserving the original hierarchical structure?
• RQ5 What theoretical guarantees can be provided for FDR control under the proposed method, particularly under independence assumptions?

Key findings

• The proposed FDR-controlling procedure maintains the DAG structure among rejected hypotheses, ensuring that if a hypothesis is rejected, all its ancestors in the DAG are also rejected.
• The method achieves higher statistical power than the Benjamini-Hochberg (BH) procedure in both simulation studies and real data analysis, particularly when the underlying DAG structure is informative.
• Theoretical analysis shows that the FDR is controlled under the independence assumption, with the proof relying on bounding the expected number of false discoveries via a PFER control framework.
• A modified version of the BH procedure is shown to preserve the DAG structure, though at the cost of reduced power, demonstrating a trade-off between structure preservation and efficiency.
• The critical value function is derived using a recursive weighting scheme based on the number of descendant leaves, which ensures proper error rate control while adapting to the graph's topology.
• The framework is general and can be extended to other error rates beyond FDR, provided a corresponding PFER control procedure is available.