[Paper Review] Learning circuits with few negations

Blais, Eric, Clément L. Canonne+3 more|arXiv (Cornell University)|Oct 30, 2014

Machine Learning and Algorithms2 references6 citations

TL;DR

This paper studies the learnability of Boolean circuits with few negation gates, introducing a structural characterization that extends Markov's theorem. It presents a uniform-distribution learning algorithm for circuits with t negations running in time n^{O(2^t \sqrt{n}/\varepsilon)}, and establishes a nearly matching lower bound of 2^{\Omega(2^t \sqrt{n}/\varepsilon)} membership queries, showing the algorithm is essentially optimal for a wide range of parameters.

ABSTRACT

Monotone Boolean functions, and the monotone Boolean circuits that compute them, have been intensively studied in complexity theory. In this paper we study the structure of Boolean functions in terms of the minimum number of negations in any circuit computing them, a complexity measure that interpolates between monotone functions and the class of all functions. We study this generalization of monotonicity from the vantage point of learning theory, giving near-matching upper and lower bounds on the uniform-distribution learnability of circuits in terms of the number of negations they contain. Our upper bounds are based on a new structural characterization of negation-limited circuits that extends a classical result of A. A. Markov. Our lower bounds, which employ Fourier-analytic tools from hardness amplification, give new results even for circuits with no negations (i.e. monotone functions).

Motivation & Objective

• To understand the learnability of Boolean functions with limited negation gates, generalizing monotone functions.
• To extend Markov's classical theorem on inversion complexity to a broader class of functions with bounded alternation complexity.
• To develop a uniform-distribution learning algorithm for circuits with t negations that matches known lower bounds in terms of query complexity.
• To establish tight lower bounds on the membership query complexity of learning such circuits, even for monotone functions (t=0).

Proposed method

• Extends Markov's theorem by proving that any k-alternating Boolean function can be expressed as f(x) = h(m1(x), ..., mk(x)), where each mi is monotone and h is parity or its negation.
• Uses this structural characterization to generalize the Fourier-based learning algorithm of Bshouty and Tamon for monotone functions to circuits with t negations.
• Applies Fourier-analytic tools and hardness amplification techniques to derive lower bounds on membership query complexity.
• Employs a reduction from hard-to-learn monotone functions to hard-to-learn k-alternating functions via function composition with PAR′_{k,r}.
• Uses noise stability bounds and additive Chernoff-type concentration to show that PAR′_{k,r} is close to the parity function with small error η = e^{-k^2/(2r)}.
• Derives a contradiction by showing that a fast learning algorithm for k-alternating functions would imply a faster algorithm for hard monotone functions, violating known lower bounds.

Experimental results

Research questions

• RQ1 Can we characterize the structure of Boolean functions that are computable with a bounded number of negation gates?
• RQ2 What is the optimal query complexity for learning circuits with t negations under the uniform distribution?
• RQ3 How does the number of negations affect the learnability of Boolean functions, and can we achieve near-optimal bounds?
• RQ4 Is the learning algorithm for t-negation circuits optimal, and what lower bounds can be proven?

Key findings

• Any k-alternating Boolean function can be exactly represented as a composition of k monotone functions and a parity or its negation.
• Every function in Cn_t (functions computable with at most t negations) can be expressed as h(m1, ..., mT) with T = O(2^t), where h is parity or its negation and each mi is monotone.
• The proposed learning algorithm runs in time n^{O(2^t \sqrt{n}/\varepsilon)} for learning to error ε under the uniform distribution.
• A nearly matching lower bound of 2^{\Omega(2^t \sqrt{n}/\varepsilon)} membership queries holds for t ≤ (1/28) log n and ε ∈ [1/n^{1/12}, 1/2 - c], showing the algorithm is essentially optimal.
• For monotone functions (t=0), the lower bound matches the best-known prior bounds, and the result extends them to a broader range of accuracy levels.
• The lower bound construction shows that even for functions with a small number of alternations, learning is hard when the error is small, and the hardness scales with the number of negations.