[Paper Review] DeepProbLog: Neural Probabilistic Logic Programming

Robin Manhaeve, Sebastijan Dumančić+3 more|ORCA Online Research @Cardiff (Cardiff University)|May 28, 2018

Natural Language Processing Techniques16 references111 citations

TL;DR

DeepProbLog extends ProbLog by adding neural predicates to jointly train probabilistic logic and neural networks end-to-end, enabling symbolic and subsymbolic reasoning with program induction.

ABSTRACT

We introduce DeepProbLog, a probabilistic logic programming language that incorporates deep learning by means of neural predicates. We show how existing inference and learning techniques can be adapted for the new language. Our experiments demonstrate that DeepProbLog supports both symbolic and subsymbolic representations and inference, 1) program induction, 2) probabilistic (logic) programming, and 3) (deep) learning from examples. To the best of our knowledge, this work is the first to propose a framework where general-purpose neural networks and expressive probabilistic-logical modeling and reasoning are integrated in a way that exploits the full expressiveness and strengths of both worlds and can be trained end-to-end based on examples.

Motivation & Objective

• Extend probabilistic logic programming with neural predicates to process subsymbolic outputs as probabilistic facts
• Preserve ProbLog semantics and inference while integrating neural network components
• Enable end-to-end gradient-based training across both probabilistic logic and neural modules
• Demonstrate capabilities in symbolic/subsymbolic reasoning, program induction, and learning from examples

Proposed method

• Define neural adic disjunctions (nADs) that wrap neural network outputs as ground probabilistic facts
• Use ProbLog inference with grounding, propositionalization, and Sentential Decision Diagrams (SDDs) to compute query probabilities
• Employ algebraic ProbLog (aProbLog) with a gradient semiring to propagate gradients through probabilistic facts
• Backpropagate loss from the logical output through neural predicates to update neural networks
• Train using standard gradient-based optimizers (e.g., Adam) alongside probabilistic parameter updates
• Normalize annotated disjunctions (ADs) to maintain valid probability distributions

Experimental results

Research questions

• RQ1 Can neural networks be integrated into a probabilistic logic programming framework without losing semantics or inference efficiency?
• RQ2 Can end-to-end gradient-based learning optimize both probabilistic facts and neural network components in a unified model?
• RQ3 How well does the combination support tasks like symbolic reasoning, program induction, and learning from examples?

Key findings

• Demonstrates joint symbolic and subsymbolic reasoning with end-to-end training
• Shows capability for program induction within a ProbLog-based framework
• Achieves competitive performance on MNIST-based addition with reasoning over digits and arithmetic background knowledge
• Exhibits scalable learning for tasks requiring both perception and probabilistic reasoning
• Provides a versatile platform for learning probabilistic logic programs with neural components