[Paper Review] VAEL: Bridging Variational Autoencoders and Probabilistic Logic Programming

Eleonora Misino, Giuseppe Marra+1 more|arXiv (Cornell University)|Feb 7, 2022

Semantic Web and Ontologies4 citations

TL;DR

VAEL introduces a neuro-symbolic generative model that integrates variational autoencoders (VAEs) with probabilistic logic programming (PLP), enabling end-to-end differentiable learning where latent representations are disentangled into symbolic logical expressions. This allows generalization to unseen generation tasks via logical reasoning, achieving state-of-the-art performance with improved data efficiency and zero-shot generalization.

ABSTRACT

We present VAEL, a neuro-symbolic generative model integrating variational autoencoders (VAE) with the reasoning capabilities of probabilistic logic (L) programming. Besides standard latent subsymbolic variables, our model exploits a probabilistic logic program to define a further structured representation, which is used for logical reasoning. The entire process is end-to-end differentiable. Once trained, VAEL can solve new unseen generation tasks by (i) leveraging the previously acquired knowledge encoded in the neural component and (ii) exploiting new logical programs on the structured latent space. Our experiments provide support on the benefits of this neuro-symbolic integration both in terms of task generalization and data efficiency. To the best of our knowledge, this work is the first to propose a general-purpose end-to-end framework integrating probabilistic logic programming into a deep generative model.

Motivation & Objective

• To address the limitation of standard VAEs in generalizing to new generation tasks without retraining by decoupling latent representations from task-specific labels.
• To integrate probabilistic logic programming as a structured, interpretable prior in the latent space to enable logical reasoning over symbolic representations.
• To enable end-to-end differentiable training that supports both neural learning and symbolic inference.
• To improve data efficiency and generalization by leveraging logical inductive bias in the latent space.
• To provide a general-purpose framework for neuro-symbolic generation that supports zero-shot adaptation to new tasks via new logical programs.

Proposed method

• Introduces a hybrid model where a VAE learns a non-linear mapping from raw input data to a symbolic latent space composed of logical atoms.
• Employs a probabilistic logic program (based on ProbLog) to define a structured prior over the symbolic latent variables, enabling logical reasoning and inference.
• Uses end-to-end backpropagation through both the VAE encoder/decoder and the logical program, enabling joint optimization of neural and symbolic components.
• Applies success probability and evidence-based sampling in the logic program to compute label probabilities from symbolic representations.
• Decouples the label prediction from the latent code by conditioning on logical expressions derived from the VAE’s latent output.
• Supports zero-shot generation by plugging in new logical programs over the same symbolic latent space without retraining the VAE.

Experimental results

Research questions

• RQ1 Can a generative model be trained to disentangle latent representations into interpretable symbolic logic expressions that support reasoning?
• RQ2 To what extent does integrating probabilistic logic programming improve generalization to unseen generation tasks compared to standard VAEs?
• RQ3 Does the symbolic prior enhance data efficiency in low-data regimes for image generation tasks?
• RQ4 Can the model generalize to new tasks by only modifying the logical program, without retraining the neural component?
• RQ5 How does the integration of symbolic reasoning improve interpretability and inductive bias in deep generative models?

Key findings

• VAEL achieves state-of-the-art generation performance on image generation benchmarks, particularly in low-data regimes.
• The model generalizes to unseen generation tasks (e.g., new arithmetic operations) by reusing the same VAE encoder with new logical programs, without retraining.
• VAEL demonstrates improved data efficiency, outperforming standard VAEs when training data is scarce.
• The integration of probabilistic logic programming enables zero-shot generalization to new reasoning tasks through symbolic program adaptation.
• The symbolic latent space supports interpretable reasoning, with success probabilities and evidence-based sampling enabling logical inference over generated content.
• To the best of the authors’ knowledge, VAEL is the first end-to-end differentiable neuro-symbolic framework integrating VAEs with probabilistic logic programming for generative modeling.