[Paper Review] Embedding Symbolic Knowledge into Deep Networks

Yaqi Xie, Ziwei Xu+3 more|arXiv (Cornell University)|Sep 3, 2019

Advanced Graph Neural Networks34 references4 citations

TL;DR

This paper proposes LENSR, a graph neural network framework that embeds symbolic logical knowledge—specifically in d-DNNF and CNF forms—into deep neural networks using semantic regularization and node heterogeneity awareness. By projecting logic formulae and assignments onto a manifold via an augmented GCN, LENSR enables efficient entailment checks and improves model performance, achieving state-of-the-art results on visual relation prediction and model checking tasks.

ABSTRACT

In this work, we aim to leverage prior symbolic knowledge to improve the performance of deep models. We propose a graph embedding network that projects propositional formulae (and assignments) onto a manifold via an augmented Graph Convolutional Network (GCN). To generate semantically-faithful embeddings, we develop techniques to recognize node heterogeneity, and semantic regularization that incorporate structural constraints into the embedding. Experiments show that our approach improves the performance of models trained to perform entailment checking and visual relation prediction. Interestingly, we observe a connection between the tractability of the propositional theory representation and the ease of embedding. Future exploration of this connection may elucidate the relationship between knowledge compilation and vector representation learning.

Motivation & Objective

• To improve deep neural network performance by integrating prior symbolic knowledge in the form of logical formulae.
• To address the challenge of scalable and effective injection of symbolic knowledge into neural networks, especially when labeled data is scarce.
• To explore whether the tractability of logical representations (e.g., d-DNNF vs. CNF) influences the learnability of vector embeddings.
• To develop a method that preserves semantic structure during embedding, ensuring embeddings reflect logical entailment and satisfiability.
• To provide a practical, trainable framework that can be used to regularize deep learning models with symbolic constraints.

Proposed method

• Proposes a graph embedding network based on Graph Convolutional Networks (GCNs) to represent propositional formulae and assignments as vectors on a manifold.
• Introduces node heterogeneity awareness in the GCN to distinguish between AND and OR nodes in logic graphs, improving structural fidelity.
• Applies semantic regularization that enforces soft constraints on the embeddings of children nodes in AND and OR nodes to preserve logical semantics.
• Uses the resulting embeddings to form logic-aware loss functions that regularize deep neural networks during training.
• Employs d-DNNF and CNF as target knowledge representations due to their contrasting tractability and size properties.
• Introduces the concept of 'embeddable-demanding' to formally compare the learnability of different logical normal forms.

Experimental results

Research questions

• RQ1 Can symbolic logical knowledge expressed in propositional logic be effectively embedded into deep neural networks using graph-based methods?
• RQ2 Does incorporating semantic regularization and node heterogeneity in GCN-based embeddings improve the alignment between vector representations and logical semantics?
• RQ3 How does the choice of logical normal form (d-DNNF vs. CNF) affect the performance and learnability of embedded representations?
• RQ4 Is there a measurable relationship between the tractability of a logical representation and its amenability to vector embedding learning?
• RQ5 Can logic-embedded representations improve downstream deep learning tasks such as visual relation prediction and entailment checking?

Key findings

• LENSR with d-DNNF and semantic regularization achieved a top-5 accuracy of 92.77% on the visual relation prediction task, significantly outperforming the baseline (84.30%) and other baselines including treeLSTM (85.76%).
• The model using d-DNNF with both heterogeneous node embedding and semantic regularization achieved 92.77% top-5 accuracy, demonstrating the effectiveness of structural and semantic constraints.
• LENSR with CNF and semantic regularization achieved 85.70% top-5 accuracy, showing improvement over the baseline but less than with d-DNNF.
• The direct application of d-DNNF via semantic loss alone only marginally improved performance (84.76%), indicating that embedding-based regularization is more effective than raw logic loss.
• The framework outperformed a state-of-the-art TreeLSTM-based embedder, highlighting the advantage of GCN-based, structure-aware embeddings over RNN-based approaches.
• The empirical results suggest a strong connection between the tractability of a logical form and its embeddability, with d-DNNF being more amenable to effective vector representation learning than CNF.