[Paper Review] Universal Hypothesis Testing with Kernels: Asymptotically Optimal Tests for Goodness of Fit

Shengyu Zhu, Biao Chen+2 more|arXiv (Cornell University)|Feb 21, 2018

Statistical Methods and Inference50 references3 citations

TL;DR

This paper establishes the first asymptotically optimal nonparametric goodness-of-fit tests for universal hypothesis testing on general sample spaces like ℝᵈ using kernel methods. It proves that Maximum Mean Discrepancy (MMD) and Kernel Stein Discrepancy (KSD) based tests achieve the optimal exponential decay rate of type-II error probability under appropriate constraints, using large deviation theory and weak metrizability of kernel discrepancies.

ABSTRACT

We characterize the asymptotic performance of nonparametric goodness of fit testing. The exponential decay rate of the type-II error probability is used as the asymptotic performance metric, and a test is optimal if it achieves the maximum rate subject to a constant level constraint on the type-I error probability. We show that two classes of Maximum Mean Discrepancy (MMD) based tests attain this optimality on $\mathbb R^d$, while the quadratic-time Kernel Stein Discrepancy (KSD) based tests achieve the maximum exponential decay rate under a relaxed level constraint. Under the same performance metric, we proceed to show that the quadratic-time MMD based two-sample tests are also optimal for general two-sample problems, provided that kernels are bounded continuous and characteristic. Key to our approach are Sanov's theorem from large deviation theory and the weak metrizable properties of the MMD and KSD.

Motivation & Objective

• To resolve the long-standing open problem of statistical optimality for nonparametric goodness-of-fit tests in universal hypothesis testing.
• To characterize the asymptotic performance of kernel-based tests using the exponential decay rate of type-II error probability as the key metric.
• To establish optimality of MMD and KSD-based tests under level constraints, particularly in high-dimensional and non-finite sample spaces.
• To extend optimality results to two-sample testing settings where sample sizes scale similarly, under bounded continuous and characteristic kernels.

Proposed method

• Uses Sanov’s theorem from large deviation theory to analyze the asymptotic behavior of the acceptance region of kernel-based tests.
• Applies weak metrizability properties of MMD and KSD to directly analyze test regions without relying on test statistics as intermediates.
• Proposes a simple MMD-based test comparing the empirical distribution of samples to the model distribution $P$ using a threshold derived via Monte Carlo sampling.
• Adapts the two-sample test framework by drawing $\omega(n)$ samples from $P$ to transform the goodness-of-fit problem into a two-sample problem.
• Relaxes the type-I error constraint to an asymptotic one for KSD-based tests to achieve optimal exponential decay rates under additional regularity conditions.
• Employs bootstrap and wild bootstrap methods to estimate critical thresholds for empirical evaluation, with kernel bandwidths selected via median heuristic or grid search.

Experimental results

Research questions

• RQ1 Can nonparametric goodness-of-fit tests achieve the same exponential decay rate of type-II error as in the simple hypothesis testing case where the alternative distribution is known?
• RQ2 Do MMD-based tests achieve asymptotic optimality in universal hypothesis testing on general Polish spaces such as ℝᵈ?
• RQ3 Can KSD-based tests achieve optimal exponential decay rates under a relaxed type-I error constraint in the universal setting?
• RQ4 Is the quadratic-time MMD-based two-sample test optimal for general two-sample problems when sample sizes scale similarly and kernels are bounded continuous and characteristic?

Key findings

• Two classes of MMD-based tests achieve the optimal exponential decay rate of type-II error probability for universal hypothesis testing on ℝᵈ, under a constant level constraint on type-I error.
• Quadratic-time KSD-based tests achieve the maximum exponential decay rate under an asymptotic level constraint, provided additional regularity conditions are satisfied.
• The quadratic-time MMD-based two-sample test is asymptotically optimal for general two-sample problems when the sample sizes scale in the same order, with the decay rate independent of the choice of bounded continuous and characteristic kernels.
• The optimality of MMD and KSD tests is established via large deviation analysis using Sanov’s theorem and the weak metrizability of kernel discrepancies.
• Empirical results show that MMD and KSD tests maintain type-I error close to the nominal level $\alpha=0.1$, with type-II error performance sensitive to kernel bandwidth and sample size.
• While finite-sample performance varies with kernel choice and distribution, the theoretical optimality holds in the large-sample limit, confirming the superiority of kernel methods in asymptotic efficiency.