[Paper Review] Local False Discovery Rate Based Methods for Multiple Testing of One-Way Classified Hypotheses

Sanat K. Sarkar, Zhigen Zhao|arXiv (Cornell University)|Dec 13, 2017

Statistical Methods in Clinical Trials30 references10 citations

TL;DR

This paper proposes two novel local false discovery rate (Lfdr)-based methods for multiple testing of one-way classified hypotheses, incorporating group structure through a grouping effect parameter. It enhances power by modeling group-specific Lfdr as a function of within-group and group-level Lfdr, achieving optimal FDR control across all hypotheses and in selective inference settings.

ABSTRACT

This paper continues the line of research initiated in Liu et. al. (2016) on developing a novel framework for multiple testing of hypotheses grouped in a one-way classified form using hypothesis-specific local false discovery rates (Lfdr's). It is built on an extension of the standard two-class mixture model from single to multiple groups, defining hypothesis-specific Lfdr as a function of the conditional Lfdr for the hypothesis given that it is within an important group and the Lfdr for the group itself and involving a new parameter that measures grouping effect. This definition captures the underlying group structure for the hypotheses belonging to a group more effectively than the standard two-class mixture model. Two new Lfdr based methods, possessing meaningful optimalities, are produced in their oracle forms. One, designed to control false discoveries across the entire collection of hypotheses, is proposed as a powerful alternative to simply pooling all the hypotheses into a single group and using commonly used Lfdr based method under the standard single-group two-class mixture model. The other is proposed as an Lfdr analog of the method of Benjamini and Bogomolov (2014) for selective inference. It controls Lfdr based measure of false discoveries associated with selecting groups concurrently with controlling the average of within-group false discovery proportions across the selected groups. Simulation studies and real-data application show that our proposed methods are often more powerful than their relevant competitors.

Motivation & Objective

• Address the challenge of controlling false discoveries in large-scale multiple testing when hypotheses are naturally grouped into one-way classified families.
• Improve statistical power over standard single-group Lfdr methods by explicitly modeling the group structure and its influence on hypothesis significance.
• Develop a Bayesian/empirical Bayes framework that incorporates both within-group and group-level Lfdr to better capture dependence and sparsity in signals.
• Provide an Lfdr analog to Benjamini and Bogomolov's (2014) method for selective inference, controlling the average false discovery proportion across selected groups.
• Bridge the gap in Lfdr-based methodologies for grouped hypotheses, particularly in settings where group importance and signal sparsity are critical.

Proposed method

• Extend the standard two-class mixture model to a multi-group framework by defining hypothesis-specific Lfdr as a function of conditional within-group Lfdr and group-level Lfdr, incorporating a new grouping effect parameter.
• Formulate two oracle-level Lfdr-based procedures: one for controlling overall FDR across all hypotheses while accounting for group structure, and another for selective inference with control over average within-group FDP across selected groups.
• Use a hierarchical Bayes approach with a Gibbs sampler to estimate model parameters, including group-specific Lfdr and the grouping effect, enabling empirical application.
• Derive key equations for posterior probabilities of null status given group-level and individual test statistics, using conditional Lfdr and product of individual Lfdr values.
• Establish theoretical optimality of the proposed methods by proving that they minimize the posterior expected false non-rejection rate (PFNR) under FDR control constraints.
• Integrate sparsity of signals both across and within groups into the Lfdr estimation, enhancing sensitivity to truly significant hypotheses in sparse settings.

Experimental results

Research questions

• RQ1 How can group structure in one-way classified hypotheses be effectively captured in Lfdr-based multiple testing to improve power over single-group pooling?
• RQ2 What is the optimal way to define group-specific Lfdr that accounts for both within-group and group-level evidence while incorporating a grouping effect parameter?
• RQ3 How can Lfdr-based methods be adapted to control the average false discovery proportion across selected groups in a selective inference framework?
• RQ4 Can an empirical Bayes approach be developed to estimate the grouping effect and Lfdr parameters in a high-dimensional, grouped multiple testing setting?
• RQ5 What is the theoretical optimality of Lfdr-based procedures in grouped settings, particularly in minimizing false non-rejections under FDR control?

Key findings

• The proposed method for overall FDR control achieves higher statistical power than standard single-group Lfdr methods by leveraging group structure and the grouping effect parameter.
• The method for selective inference controls the average false discovery proportion across selected groups while maintaining optimal control over the overall FDR, outperforming existing approaches in this context.
• Simulation studies demonstrate that the proposed methods consistently outperform competitors in terms of power and FDR control, especially when signals are sparse across and within groups.
• The Gibbs sampler with hierarchical Bayes estimation successfully recovers the true grouping effect and Lfdr parameters, enabling reliable empirical application.
• Application to the Adequate Year Progress (AYP) data set confirms the method's practical utility and superior performance in real-world settings.
• Theoretical results show that the proposed procedures are optimal in minimizing the posterior expected false non-rejection rate (PFNR) under FDR control, establishing their decision-theoretic optimality.