[Paper Review] Predicate Invention by Learning From Failures

Andrew Cropper, Rolf Morel|arXiv (Cornell University)|Apr 29, 2021

Logic, Reasoning, and Knowledge31 references6 citations

TL;DR

This paper introduces Poppi, an inductive logic programming system that enables automatic predicate invention by learning from failures using answer set programming. It outperforms existing ILP systems by discovering useful, compact predicates without requiring user-defined metarules or manual symbol specification, significantly improving learning performance when beneficial and remaining efficient when not needed.

ABSTRACT

Discovering novel high-level concepts is one of the most important steps needed for human-level AI. In inductive logic programming (ILP), discovering novel high-level concepts is known as predicate invention (PI). Although seen as crucial since the founding of ILP, PI is notoriously difficult and most ILP systems do not support it. In this paper, we introduce POPPI, an ILP system that formulates the PI problem as an answer set programming problem. Our experiments show that (i) PI can drastically improve learning performance when useful, (ii) PI is not too costly when unnecessary, and (iii) POPPI can substantially outperform existing ILP systems.

Motivation & Objective

• To address the long-standing challenge of predicate invention (PI) in inductive logic programming (ILP), which is crucial for human-level AI but rarely supported in practice.
• To overcome the limitations of existing PI methods that require user-provided metarules or manual specification of predicate arity and types.
• To develop a scalable, automatic PI method that integrates seamlessly into the learning process and improves hypothesis compactness and performance.
• To demonstrate that PI can be both highly beneficial when useful and cost-effective when unnecessary.

Proposed method

• Formulates the predicate invention problem as an answer set programming (ASP) problem to enable declarative, scalable reasoning.
• Extends the learning-from-failures (LFF) framework with a new type of redundancy constraint that is sound with respect to optimal solutions.
• Automatically infers predicate arity and argument types during the learning process, eliminating the need for user-defined symbols or metarules.
• Uses failure analysis to generate hypothesis constraints that prune the search space, improving efficiency and convergence.
• Integrates PI within the core learning loop, allowing simultaneous discovery of predicates and hypotheses.
• Employs a three-stage cycle: generate, test, and constrain, where failures guide the refinement of candidate hypotheses and invented predicates.

Experimental results

Research questions

• RQ1 Can predicate invention significantly improve learning performance in ILP when useful?
• RQ2 Is the cost of predicate invention negligible when it is not needed?
• RQ3 Can an ILP system with automatic predicate invention outperform existing systems that rely on user-provided metarules?
• RQ4 How effective is the proposed redundancy constraint in pruning the hypothesis space while preserving optimal solutions?
• RQ5 Can the system learn recursive, multi-clause programs with automatically invented predicates?

Key findings

• Poppi achieved 100% predictive accuracy on all programming puzzles where it succeeded, including complex tasks like finding duplicate elements and sorting lists.
• On the finddup task, Poppi achieved 98% accuracy with a standard error of 0, significantly outperforming Metagol’s 50% accuracy.
• Poppi learned solutions in under 1 second on most tasks, with the longest learning time being 91 seconds on the finddup task, compared to Popper’s 48 seconds on the same task.
• Poppi’s learning time was consistently higher than Popper’s, but the difference was always less than double, indicating that PI is not prohibitively expensive when not useful.
• Poppi subsumes Metagol’s capabilities and can replicate its search strategy, demonstrating that it is a more general and powerful framework.
• In experiments, Poppi outperformed Metagol on all but three problems, with identical performance on those three, confirming its superior scalability and effectiveness.