[Paper Review] On K-wise Independent Distributions and Boolean Functions

Derksen, Harm, Ivanov, Peter+2 more|arXiv (Cornell University)|Jan 16, 2012

Complexity and Algorithms in Graphs38 references22 citations

TL;DR

This paper investigates how k-wise independent distributions—where every k bits are independent—can 'fool' monotone Boolean functions that exhibit sharp threshold behavior. Using tools from linear programming duality, moment problems, and error-correcting codes, the authors show that for functions like Majority and AND, the required k to preserve threshold behavior (K2) is significantly higher than for off-critical behavior (K1), with tight bounds derived via moment-based inequalities and orthogonal array constructions.

ABSTRACT

We prove several new results about bounded uniform and small-bias distributions. A main message is that, small-bias, even perturbed with noise, does not fool several classes of tests better than bounded uniformity. We prove this for threshold tests, small-space algorithms, and small-depth circuits. In particular, we obtain small-bias distributions that - achieve an optimal lower bound on their statistical distance to any bounded-uniform distribution. This closes a line of research initiated by Alon, Goldreich, and Mansour in 2003, and improves on a result by O'Donnell and Zhao. - have heavier tail mass than the uniform distribution. This answers a question posed by several researchers including Bun and Steinke. - rule out a popular paradigm for constructing pseudorandom generators, originating in a 1989 work by Ajtai and Wigderson. This again answers a question raised by several researchers. For branching programs, our result matches a bound by Forbes and Kelley. Our small-bias distributions above are symmetric. We show that the xor of any two symmetric small-bias distributions fools any bounded function. Hence our examples cannot be extended to the xor of two small-bias distributions, another popular paradigm whose power remains unknown. We also generalize and simplify the proof of a result of Bazzi.

Motivation & Objective

• Understand how k-wise independent distributions affect the behavior of monotone Boolean functions that exhibit sharp threshold transitions.
• Distinguish between K1 (independence level needed to preserve threshold existence) and K2 (independence level needed to preserve behavior at the critical probability pc).
• Characterize the extremal properties of k-wise independent distributions, especially the maximal and minimal probabilities of all bits being 1.
• Establish connections between k-wise independence and polynomial approximation, particularly a stronger 'sandwich L1' norm approximation.
• Provide new bounds on the size of orthogonal arrays using moment problem techniques, improving known results for GF(q)-valued designs.

Proposed method

• Use duality in linear programming to equate the problem of fooling Boolean functions under k-wise independence to approximating the function by a real polynomial in a 'sandwich L1' norm.
• Apply theorems from the classical moment problem (TCMP) to bound the maximal and minimal probabilities of all-ones events under k-wise independent distributions.
• Derive bounds on M(n,k,p) = max_Q Q(All bits are 1) using moment matching: if S is the number of 1s, then E[S^j] = E[Bin(n,p)^j] for j ≤ k.
• Construct k-wise independent distributions using linear subspaces and error-correcting code constructions (e.g., (u|u+v) construction), enabling tailored behavior for specific functions.
• Use the Rao bound and Gilbert-Varshamov bound from coding theory to derive lower and upper bounds on the size of orthogonal arrays.
• Analyze specific functions (AND, Majority, Tribes, Percolation) to show that noise stability does not guarantee robustness under k-wise independence, even for growing k.

Experimental results

Research questions

• RQ1 What is the minimal k (K2) such that any k-wise independent distribution with marginal p = pc makes a monotone Boolean function f behave nearly as under full independence?
• RQ2 How does K2 compare to K1, the minimal k needed to preserve the existence of a sharp threshold?
• RQ3 Can the maximal probability that all n bits are 1 under k-wise independent distributions be bounded tightly, and what does this imply for orthogonal array constructions?
• RQ4 Is there a fundamental difference between general k-wise independent distributions and those derived from linear orthogonal arrays (e.g., in terms of extremal probabilities) ?
• RQ5 To what extent does noise stability of a function imply robustness under k-wise independent inputs, even as k grows with n?

Key findings

• For the Majority function, K1 = 2 (2-wise independence suffices to preserve behavior when p < 1/2), but K2 = ω(1) (unbounded k is needed at p = 1/2), with |P(Maj=1) − 1/2| ≤ O(1/√k).
• The maximal probability that all bits are 1 under k-wise independent distributions is bounded by M(n,k,p) ≤ pn / P(Bin(n,1−p) ≤ k/2), with tighter bounds like M(n,k,p) ≤ 2√k (kp/(2e(1−p)(n−k/2)))^{k/2} for even k.
• For p = 1/q with prime power q, the bound on M(n,k,p) yields a new lower bound on the size of orthogonal arrays, improving the classical Rao bound in some cases.
• There exists a distribution in A(n,3,1/3) with P(all ones) = Ω(1/n), while all linear orthogonal array-based constructions yield P(all ones) = O(1/(n log n)), showing a fundamental gap between general and linear k-wise independent distributions.
• For p ≥ 1/2, the minimal n such that m(n,k,p) = 0 (i.e., no k-wise independent distribution can have P(all ones) > 0) satisfies nc(k,p) ≥ k/(2(1−p)) for even k, and the upper bound nc(k,p) ≤ C k / ((1−p) log(1/(1−p))) is derived using the Gilbert-Varshamov bound and TCMP.
• The paper shows that even noise-stable functions (e.g., iterated Majority of threes) can behave very differently under k-wise independent inputs than under full independence, even when k grows with n, challenging the intuition that stability implies robustness.