[Paper Review] Multiple Testing under Dependence with Approximate Conditional Likelihood

Sairam Rayaprolu, Zhiyi Chi|arXiv (Cornell University)|Dec 25, 2014

Statistical Methods in Clinical Trials55 references3 citations

TL;DR

This paper proposes a novel method for multiple testing under dependence by approximating conditional likelihoods of signal states using nearby observations and estimated low-order moments from noiseless signal samples. It achieves stable false discovery rate control in stationary ergodic binary signal processes with known noise, demonstrating robustness and power through simulations using hidden Markov models.

ABSTRACT

Statistical dependence in data poses a significant challenge to the stability of large scale multiple testing. This makes the variances of the number of true discoveries and the false discovery proportion crucial considerations in the false discovery rate control [15, 30, 31]. However, the statistical dependence structure of data is often unknown. We address the problem for the case of a stationary ergodic binary signal process embedded in noisy observations, where the distribution of the noise is known while that of the signal process is unknown. A novel aspect of our approach is the approximation of the conditional likelihoods of signals at individual sites given the data by incorporating nearby observations as well as estimates of low order moments obtained from a noiseless sample of the signal process. Simulations are carried out to show the stability as well as validity and power of the approach. The simulations rely on sampling of hidden Markov dependence structures as random matrices with specified stationary distribution and lower bounds on the top absolute eigenvalues, which is of interest in its own right.

Motivation & Objective

• Address the challenge of unstable false discovery rate (FDR) control in multiple testing when data exhibit unknown statistical dependence.
• Develop a method that maintains validity and power under dependence by leveraging nearby observations and moment estimates from noiseless signal samples.
• Handle the case of a stationary ergodic binary signal process corrupted by known noise, where the signal distribution is unknown.
• Ensure robustness of FDR control by approximating conditional likelihoods that incorporate local data structure and empirical signal moments.

Proposed method

• Approximate the conditional likelihood of signal states at each site using neighboring observations to capture local dependence structure.
• Incorporate estimated low-order moments of the signal process derived from a noiseless sample to improve likelihood approximation.
• Model the signal process as a stationary ergodic binary process with unknown distribution but known noise distribution.
• Use hidden Markov models to simulate dependence structures, specifying stationary distributions and lower bounds on the top absolute eigenvalues.
• Construct test statistics based on the approximated conditional likelihoods to assess significance under dependence.
• Validate the method through simulations that generate random matrices with controlled stationary distributions and spectral properties.

Experimental results

Research questions

• RQ1 How can conditional likelihoods be approximated in multiple testing when the signal distribution is unknown but the noise distribution is known?
• RQ2 To what extent does incorporating local observations and empirical signal moments improve FDR control under dependence?
• RQ3 What is the impact of hidden Markov dependence structures on the stability and power of multiple testing procedures?
• RQ4 Can the method maintain valid FDR control while achieving high statistical power in the presence of unknown dependence?
• RQ5 How do lower bounds on the top absolute eigenvalues of the dependence matrix affect the performance of the testing procedure?

Key findings

• The proposed method achieves stable false discovery rate control even under complex dependence structures, as demonstrated in simulations.
• Incorporating nearby observations and estimated signal moments significantly improves the accuracy of conditional likelihood approximation.
• The method maintains high statistical power across various dependence scenarios, particularly when the signal process exhibits Markovian dependence.
• Simulations using hidden Markov models with specified stationary distributions and eigenvalue constraints confirm the method's robustness.
• The approach remains valid under unknown signal distributions as long as the noise distribution is known and the signal process is stationary and ergodic.
• The use of noiseless signal samples to estimate low-order moments enhances the stability of the test procedure without requiring full knowledge of the signal distribution.