[Paper Review] Faster Algorithms for Testing under Conditional Sampling

Moein Falahatgar, Ashkan Jafarpour+3 more|arXiv (Cornell University)|Apr 16, 2015

Machine Learning and Algorithms1 references3 citations

TL;DR

This paper presents faster algorithms for distribution property testing under the conditional sampling model, where queries return samples from user-specified subsets of the domain. It achieves optimal sample complexity of $\widetilde{\mathcal{O}}(\epsilon^{-2})$ for identity testing and sub-logarithmic $\widetilde{\mathcal{O}}(\epsilon^{-5}\log\log k)$ for closeness testing, significantly improving prior bounds and matching information-theoretic limits.

ABSTRACT

There has been considerable recent interest in distribution-tests whose run-time and sample requirements are sublinear in the domain-size $k$. We study two of the most important tests under the conditional-sampling model where each query specifies a subset $S$ of the domain, and the response is a sample drawn from $S$ according to the underlying distribution. For identity testing, which asks whether the underlying distribution equals a specific given distribution or $ε$-differs from it, we reduce the known time and sample complexities from $ ilde{\mathcal{O}}(ε^{-4})$ to $ ilde{\mathcal{O}}(ε^{-2})$, thereby matching the information theoretic lower bound. For closeness testing, which asks whether two distributions underlying observed data sets are equal or different, we reduce existing complexity from $ ilde{\mathcal{O}}(ε^{-4} \log^5 k)$ to an even sub-logarithmic $ ilde{\mathcal{O}}(ε^{-5} \log \log k)$ thus providing a better bound to an open problem in Bertinoro Workshop on Sublinear Algorithms [Fisher, 2004].

Motivation & Objective

• To reduce the sample and time complexity of identity and closeness testing under the conditional sampling model.
• To close the gap between known upper bounds and information-theoretic lower bounds for identity testing.
• To resolve an open problem in sublinear algorithms concerning the complexity of closeness testing.
• To develop efficient algorithms that achieve sub-logarithmic dependence on domain size $k$ for closeness testing.

Proposed method

• Leverages conditional sampling queries that return samples from user-defined subsets of the domain.
• Introduces a novel recursive binary search framework to efficiently identify significant domain elements.
• Employs a pruning technique to eliminate low-probability elements and reduce search space.
• Uses a multi-level approximation strategy to estimate distribution parameters with high confidence using few samples.
• Applies a carefully designed sampling strategy with error bounds derived from chi-squared divergence and concentration inequalities.
• Combines multiple subroutines—Assisted-closeness-test and Binary-search—into a hierarchical algorithmic pipeline.

Experimental results

Research questions

• RQ1 Can identity testing be performed with sample complexity matching the information-theoretic lower bound?
• RQ2 What is the optimal sample complexity for closeness testing under conditional sampling, especially in terms of dependence on $k$?
• RQ3 Can the dependence on $k$ in closeness testing be reduced to sub-logarithmic levels?
• RQ4 How can conditional sampling be leveraged to design algorithms with significantly improved complexity over standard sampling models?

Key findings

• The sample complexity for identity testing is reduced to $\widetilde{\mathcal{O}}(\epsilon^{-2})$, matching the information-theoretic lower bound.
• For closeness testing, the sample complexity is improved to $\widetilde{\mathcal{O}}(\epsilon^{-5}\log\log k)$, achieving sub-logarithmic dependence on $k$.
• The algorithm for closeness testing resolves an open problem posed at the Bertinoro Workshop on Sublinear Algorithms (Fisher, 2014).
• The proposed algorithm achieves a success probability of at least $1/30$ when distributions are $\epsilon$-far, and at least $1-\delta$ when they are equal.
• The total sample complexity of the closeness testing algorithm is $\widetilde{\mathcal{O}}(\epsilon^{-5}\log\log k)$, improving upon the previous $\widetilde{\mathcal{O}}(\epsilon^{-4}\log^5 k)$ bound.