[Paper Review] Testing Identity of Multidimensional Histograms

Ilias Diakonikolas, Daniel M. Kane+1 more|arXiv (Cornell University)|Apr 10, 2018

Machine Learning and Algorithms56 references4 citations

TL;DR

This paper presents the first sample-optimal, computationally efficient identity tester for multidimensional $k$-histogram distributions in fixed dimension $d$, achieving sample complexity $O((\sqrt{k}/\epsilon^{2})2^{d/2}\log^{2.5d}(k/\epsilon))$ and matching nearly tight lower bounds. The algorithm is robust to model misspecification and establishes a nearly optimal separation between learning and testing complexity for this nonparametric family.

ABSTRACT

We investigate the problem of identity testing for multidimensional histogram distributions. A distribution $p: D ightarrow \mathbb{R}_+$, where $D \subseteq \mathbb{R}^d$, is called a $k$-histogram if there exists a partition of the domain into $k$ axis-aligned rectangles such that $p$ is constant within each such rectangle. Histograms are one of the most fundamental nonparametric families of distributions and have been extensively studied in computer science and statistics. We give the first identity tester for this problem with {\em sub-learning} sample complexity in any fixed dimension and a nearly-matching sample complexity lower bound. In more detail, let $q$ be an unknown $d$-dimensional $k$-histogram distribution in fixed dimension $d$, and $p$ be an explicitly given $d$-dimensional $k$-histogram. We want to correctly distinguish, with probability at least $2/3$, between the case that $p = q$ versus $\|p-q\|_1 \geq ε$. We design an algorithm for this hypothesis testing problem with sample complexity $O((\sqrt{k}/ε^2) 2^{d/2} \log^{2.5 d}(k/ε))$ that runs in sample-polynomial time. Our algorithm is robust to model misspecification, i.e., succeeds even if $q$ is only promised to be {\em close} to a $k$-histogram. Moreover, for $k = 2^{Ω(d)}$, we show a sample complexity lower bound of $(\sqrt{k}/ε^2) \cdot Ω(\log(k)/d)^{d-1}$ when $d\geq 2$. That is, for any fixed dimension $d$, our upper and lower bounds are nearly matching. Prior to our work, the sample complexity of the $d=1$ case was well-understood, but no algorithm with sub-learning sample complexity was known, even for $d=2$. Our new upper and lower bounds have interesting conceptual implications regarding the relation between learning and testing in this setting.

Motivation & Objective

• To design an efficient identity tester for $d$-dimensional $k$-histogram distributions with sub-learning sample complexity.
• To establish nearly matching upper and lower bounds on sample complexity for identity testing in fixed dimension $d$.
• To investigate the relationship between learning and testing complexity for multidimensional histogram families.
• To develop a robust algorithm that tolerates $q$ being close to, but not exactly, a $k$-histogram.
• To extend prior work beyond the $d=1$ case, where sub-learning complexity was already known, to $d \geq 2$.

Proposed method

• The algorithm uses a partitioning strategy that divides the $d$-dimensional domain into axis-aligned rectangles based on the structure of the known $k$-histogram $p$.
• It applies a modified $\ell_1$-distance estimation technique using sample access to the unknown distribution $q$ and the explicit distribution $p$.
• The method leverages concentration inequalities and chi-squared divergence analysis to bound error probabilities under both null and alternative hypotheses.
• A key component is the use of a reduction to uniformity testing on a carefully constructed partitioned domain to derive lower bounds.
• The analysis incorporates a randomized construction of adversarial distributions to prove the lower bound via information-theoretic arguments.
• The algorithm runs in sample-polynomial time and is robust to $\|q - \mathrm{histogram}\|_1 \leq \epsilon/3$.

Experimental results

Research questions

• RQ1 Can we design an identity tester for $d$-dimensional $k$-histograms with sample complexity sub-linear in the domain size and independent of dimension in the exponent?
• RQ2 What is the optimal sample complexity for identity testing of $k$-histograms in fixed dimension $d \geq 2$?
• RQ3 How does the sample complexity of identity testing compare to that of learning for $k$-histograms?
• RQ4 Can we achieve robustness to model misspecification in identity testing for multidimensional histograms?
• RQ5 What is the information-theoretic limit of identity testing for $k$-histograms when $k = 2^{\Omega(d)}$?

Key findings

• The proposed identity tester achieves sample complexity $O((\sqrt{k}/\epsilon^{2})2^{d/2}\log^{2.5d}(k/\epsilon))$ for $d$-dimensional $k$-histograms in fixed dimension $d$.
• The algorithm is robust to $q$ being $\epsilon/3$-close to a $k$-histogram, not necessarily exactly one.
• A nearly matching sample complexity lower bound of $(\sqrt{k}/\epsilon^{2}) \cdot \Omega(\log(k)/d)^{d-1}$ is proven for $k = 2^{\Omega(d)}$ and $d \geq 2$.
• The upper and lower bounds are nearly matching for any fixed dimension $d$, resolving the sample complexity up to logarithmic factors.
• The work establishes a nearly optimal separation between learning and identity testing complexity for $k$-histograms, showing testing can be significantly more efficient than learning.
• The results imply that for multidimensional histograms, identity testing does not require the full sample complexity of learning, even in high dimensions when $k$ is exponential in $d$.