[Paper Review] Probabilistic Inductive Logic Programming Based on Answer Set Programming

Matthias Nickles, Alessandra Mileo|arXiv (Cornell University)|May 4, 2014

Logic, Reasoning, and Knowledge25 references17 citations

TL;DR

This paper proposes a novel probabilistic inductive logic programming framework based on Answer Set Programming (ASP), enabling probabilistic annotation of first-order formulas and ASP rules, with semantics grounded in soft and hard constraints. It supports parameter estimation via gradient ascent with numerical differentiation, achieving accurate inference and learning even in complex relational domains with uncertainty.

ABSTRACT

We propose a new formal language for the expressive representation of probabilistic knowledge based on Answer Set Programming (ASP). It allows for the annotation of first-order formulas as well as ASP rules and facts with probabilities and for learning of such weights from data (parameter estimation). Weighted formulas are given a semantics in terms of soft and hard constraints which determine a probability distribution over answer sets. In contrast to related approaches, we approach inference by optionally utilizing so-called streamlining XOR constraints, in order to reduce the number of computed answer sets. Our approach is prototypically implemented. Examples illustrate the introduced concepts and point at issues and topics for future research.

Motivation & Objective

• To address the lack of expressive, extensible frameworks for probabilistic knowledge representation in relational domains.
• To enable probabilistic reasoning over first-order logic and ASP rules using soft and hard constraints.
• To support parameter estimation of formula weights from relational data using gradient-based optimization.
• To reduce computational cost in inference through optional streamlining XOR constraints.
• To provide a foundation for future work in structure learning and performance optimization.

Proposed method

• Annotates first-order formulas and ASP rules with probabilities to represent uncertainty.
• Assigns semantics via soft and hard constraints, defining a probability distribution over answer sets.
• Uses answer set solvers to compute stable models, with optional streamlining via XOR constraints to reduce answer set count.
• Employs numerical differentiation to approximate gradients for parameter learning, using the Barzilai-Borwein method for optimization.
• Applies gradient ascent to maximize the likelihood of observed examples given the hypothesis and background knowledge.
• Implements a prototype that supports inference and weight learning, with empirical evaluation on a localization scenario.

Experimental results

Research questions

• RQ1 How can probabilistic knowledge be effectively represented in a logic programming framework that supports relational structures and uncertainty?
• RQ2 Can answer set programming be extended to support probabilistic inference with full expressiveness for formula and rule weights?
• RQ3 How can parameter estimation be performed efficiently for probabilistic logic programs with complex relational data?
• RQ4 What impact do streamlining XOR constraints have on reducing the number of answer sets and improving inference scalability?
• RQ5 How accurate and efficient is gradient-based learning of formula weights in practice?

Key findings

• The framework supports probabilistic annotation of arbitrary first-order formulas and ASP rules without syntactic restrictions.
• Inference on a 100-point indoor localization example required approximately 2250 ms without sampling, demonstrating feasibility on moderate-sized problems.
• With 1000 points, inference became tractable only through sampling, highlighting the need for scalable techniques.
• Parameter learning for a removed formula (moved(1)) converged to Pr(moved(1)) ≈ 0 in about 3170 ms, confirming effective learning from data.
• The use of numerical differentiation with small h-values (e.g., √ε × wi) enables gradient approximation even when closed-form expressions are unavailable.
• The approach enables learning from both positive and unweighted facts, supporting flexible example specification.