[Paper Review] A regularity lemma, and low-weight approximators, for low-degree polynomial threshold functions

Ilias Diakonikolas, Rocco A. Servedio+2 more|arXiv (Cornell University)|Sep 25, 2009

Complexity and Algorithms in Graphs28 references4 citations

TL;DR

This paper presents a regularity lemma for degree-d polynomial threshold functions (PTFs) over the Boolean cube, showing that any such PTF can be decomposed into a constant-depth decision tree where most leaves correspond to PTFs that are close to regular (i.e., with balanced variable influences). Using this structure, the authors prove that every degree-d PTF admits an ǫ-approximator of constant degree with integer weights summing to O(nd), which is optimal up to logarithmic factors—resolving a key question in complexity and learning theory about low-weight approximations of PTFs.

ABSTRACT

We give a "regularity lemma" for degree-d polynomial threshold functions (PTFs) over the Boolean cube {-1,1}^n. This result shows that every degree-d PTF can be decomposed into a constant number of subfunctions such that almost all of the subfunctions are close to being regular PTFs. Here a "regular PTF is a PTF sign(p(x)) where the influence of each variable on the polynomial p(x) is a small fraction of the total influence of p. As an application of this regularity lemma, we prove that for any constants d \geq 1, \eps \geq 0, every degree-d PTF over n variables has can be approximated to accuracy eps by a constant-degree PTF that has integer weights of total magnitude O(n^d). This weight bound is shown to be optimal up to constant factors.

Motivation & Objective

• To develop a structural decomposition for low-degree polynomial threshold functions (PTFs) using a regularity lemma.
• To establish that any degree-d PTF can be approximated by a constant-degree PTF with integer weights of total magnitude O(nd).
• To prove that this weight bound is optimal up to logarithmic factors, even when allowing arbitrary-degree approximators.
• To provide a general tool for reducing problems on general PTFs to problems on regular PTFs, which are easier to analyze due to their pseudorandom-like properties.
• To extend prior results on low-weight approximators from linear threshold functions (LTFs) to higher-degree PTFs.

Proposed method

• Introduce a notion of τ-regularity for polynomials, where each variable's influence is at most a τ fraction of the total influence.
• Prove a regularity lemma that decomposes any degree-d PTF into a decision tree of depth depth(d, τ) = (1/τ) · (d log(1/τ))^{O(d)} such that most leaves correspond to PTFs τ-close to τ-regular ones.
• Use the invariance principle and anti-concentration bounds for regular polynomials to control error probabilities in approximation.
• Construct a low-weight approximator by combining regularized PTFs at good leaves and constant functions at bad leaves, using multilinear indicator polynomials Pρ to define the final PTF.
• Bound the total weight of the approximating polynomial Q by leveraging Cauchy-Schwarz and the fact that each Pρq(ρ) term has bounded ℓ2 norm.
• Establish tight lower bounds via explicit constructions showing that Ωd(nd/log n) and Ωd(nd−1) weight are necessary for ǫ-approximation, even with arbitrary-degree approximators.

Experimental results

Research questions

• RQ1 Can every degree-d PTF be decomposed into a small number of subfunctions that are close to regular PTFs?
• RQ2 Is there a constant-degree PTF with integer weights O(nd) that ǫ-approximates any degree-d PTF?
• RQ3 Can the weight bound of O(nd) be improved, or is it optimal up to logarithmic factors?
• RQ4 What is the minimal weight required to ǫ-approximate a degree-d PTF, even when allowing arbitrary-degree approximating PTFs?
• RQ5 How can regularity in variable influences be leveraged to simplify analysis and approximation of PTFs?

Key findings

• A regularity lemma is established: any degree-d PTF can be decomposed into a decision tree of depth (1/τ) · (d log(1/τ))^{O(d)} such that with probability at least 1−τ, a random input reaches a leaf whose PTF is τ-close to a τ-regular PTF.
• Every degree-d PTF over n variables admits an ǫ-approximator that is a constant-degree PTF with integer weights summing to O(nd), which is optimal up to logarithmic factors.
• The weight bound of O(nd) is shown to be tight: there exist degree-d PTFs requiring Ωd(nd/log n) weight for ǫ-approximation by constant-degree PTFs.
• Even when arbitrary-degree approximators are allowed, some degree-d PTFs require Ωd(nd−1) weight for ǫ-approximation, showing limited room for improvement.
• The proof technique relies on combining regularity decomposition with anti-concentration bounds and ℓ2 norm estimates to control approximation error.
• The result generalizes prior work on low-weight approximators for linear threshold functions to higher-degree PTFs, unifying structural and approximation-theoretic insights.