[Paper Review] Understanding Boolean Function Learnability on Deep Neural Networks.

Anderson Rocha Tavares, Pedro H. C. Avelar+4 more|arXiv (Cornell University)|Sep 13, 2020

Machine Learning and Algorithms50 references4 citations

TL;DR

This paper investigates how deep neural networks learn boolean functions derived from combinatorial optimization problems, model sampling benchmarks, and random 3-CNF formulas. It finds that small, shallow networks effectively approximate these functions, with underconstrained 3-CNFs being harder to learn than overconstrained ones, and smaller formulas posing greater challenges due to fewer positive examples.

ABSTRACT

Computational learning theory states that many classes of boolean formulas are learnable in polynomial time. This paper addresses the understudied subject of how, in practice, such formulas can be learned by deep neural networks. Specifically, we analyse boolean formulas associated with the decision version of combinatorial optimisation problems, model sampling benchmarks, and random 3-CNFs with varying degrees of constrainedness. Our extensive experiments indicate that: (i) regardless of the combinatorial optimisation problem, relatively small and shallow neural networks are very good approximators of the associated formulas; (ii) smaller formulas seem harder to learn, possibly due to the fewer positive (satisfying) examples available; and (iii) interestingly, underconstrained 3-CNF formulas are more challenging to learn than overconstrained ones. Source code and relevant datasets are publicly available (this https URL).

Motivation & Objective

• To investigate the practical learnability of boolean functions from combinatorial optimization problems using deep neural networks.
• To analyze the impact of formula size and constrainedness on neural network learning performance.
• To evaluate the effectiveness of small, shallow networks in approximating boolean formulas derived from real-world and synthetic benchmarks.
• To understand why certain boolean formulas—especially smaller or underconstrained 3-CNFs—are more difficult to learn.

Proposed method

• Training small, shallow feedforward neural networks on labeled boolean formula instances derived from decision versions of combinatorial optimization problems.
• Using datasets of model sampling benchmarks and random 3-CNF formulas with varying degrees of constrainedness.
• Evaluating generalization performance using standard classification metrics on test sets of satisfying and unsatisfying assignments.
• Systematically varying formula size and constrainedness to assess their impact on learning accuracy and convergence.
• Employing fixed network architectures across experiments to isolate the effects of input formula characteristics.

Experimental results

Research questions

• RQ1 Can small, shallow neural networks effectively approximate boolean functions from combinatorial optimization problems?
• RQ2 How does the size of a boolean formula affect its learnability by deep neural networks?
• RQ3 Does the degree of constrainedness in random 3-CNF formulas influence the difficulty of learning the associated boolean function?
• RQ4 Why are underconstrained 3-CNF formulas more challenging to learn than overconstrained ones?

Key findings

• Small and shallow neural networks achieve strong approximation performance on boolean formulas from combinatorial optimization problems.
• Smaller boolean formulas are harder to learn, likely due to fewer positive (satisfying) examples in the training data.
• Underconstrained 3-CNF formulas are more difficult to learn than overconstrained ones, contrary to intuitive expectations.
• The generalization performance of neural networks is significantly influenced by the distribution of satisfying assignments in the training data.