[Paper Review] Logical Entity Representation in Knowledge-Graphs for Differentiable Rule Learning

Chi Han, Qizheng He+4 more|arXiv (Cornell University)|May 22, 2023

Data Quality and ManagementDecision Sciences3 citations

TL;DR

This paper proposes Logical Entity RePresentation (LERP), a differentiable, interpretable vector of probabilistic logical functions that encodes contextual subgraph information around entities in knowledge graphs. By integrating LERP into probabilistic logical rule learning, the model learns more expressive rules, achieving state-of-the-art performance on knowledge graph completion benchmarks and outperforming both rule-based baselines and black-box methods like TransE, while maintaining interpretability through explicit logical rules.

ABSTRACT

Probabilistic logical rule learning has shown great strength in logical rule mining and knowledge graph completion. It learns logical rules to predict missing edges by reasoning on existing edges in the knowledge graph. However, previous efforts have largely been limited to only modeling chain-like Horn clauses such as $R_1(x,z)\land R_2(z,y)\Rightarrow H(x,y)$. This formulation overlooks additional contextual information from neighboring sub-graphs of entity variables $x$, $y$ and $z$. Intuitively, there is a large gap here, as local sub-graphs have been found to provide important information for knowledge graph completion. Inspired by these observations, we propose Logical Entity RePresentation (LERP) to encode contextual information of entities in the knowledge graph. A LERP is designed as a vector of probabilistic logical functions on the entity's neighboring sub-graph. It is an interpretable representation while allowing for differentiable optimization. We can then incorporate LERP into probabilistic logical rule learning to learn more expressive rules. Empirical results demonstrate that with LERP, our model outperforms other rule learning methods in knowledge graph completion and is comparable or even superior to state-of-the-art black-box methods. Moreover, we find that our model can discover a more expressive family of logical rules. LERP can also be further combined with embedding learning methods like TransE to make it more interpretable.

Motivation & Objective

• To address the limitation of existing rule learning methods that only model chain-like Horn clauses and ignore contextual subgraph information around entities.
• To develop a differentiable, interpretable representation that captures logical patterns from local subgraphs of entities in knowledge graphs.
• To improve knowledge graph completion by incorporating contextual relational structure into probabilistic logical rule learning.
• To enable hybrid learning by combining LERP with embedding models like TransE for more interpretable and generalizable representations.

Proposed method

• LERP represents each entity as a vector of probabilistic logical functions over its neighboring subgraph, such as ∃z′ is_son_of(y,z′).
• The method formulates logical rules using LERP components, enabling differentiable optimization over both rules and representations.
• A joint training framework optimizes LERP and rule weights end-to-end using a differentiable loss function with regularization.
• The model integrates LERP with TransE via a hybrid architecture: Embed′(e) = Embed(e) + R·L^Θ(e), where R maps LERP to embedding space.
• Regularization terms, including ℓ2 norm and entropy regularization on rule weights, are introduced to encourage sparsity and interpretability.
• The method supports interpretability by linking learned logical functions to entity relations and embedding vectors through matrix projection (Moore–Penrose inverse).

Figure 1: Incorporating contextual information of entities can benefit missing link prediction.

Experimental results

Research questions

• RQ1 Can incorporating local subgraph context into entity representations improve the expressiveness of logical rule learning in knowledge graphs?
• RQ2 Can a differentiable, interpretable entity representation like LERP outperform existing rule learning and black-box embedding models in knowledge graph completion?
• RQ3 How can LERP be effectively combined with neural embedding models like TransE to enhance interpretability and generalization?
• RQ4 What kind of logical rules does LERP enable the model to discover, and how do they relate to real-world relational patterns?

Key findings

• The proposed model with LERP outperforms existing rule learning baselines on knowledge graph completion benchmarks, achieving higher AUC and MRR scores.
• LERP enables the model to discover a more expressive family of logical rules, including complex relational patterns involving existential quantifiers.
• The hybrid TransE+LERP model achieves better generalization than vanilla TransE, especially on small datasets like Family, due to logical regularization from LERP.
• Adding entropy regularization to LERP improves performance further by encouraging sparsity and explicitness in learned logical functions.
• The model's interpretability is validated by linking learned logical functions to specific relations (e.g., niece, daughter) and showing their alignment with semantic meaning.
• Empirical results show that LERP can be effectively combined with TransE, resulting in a model that is both accurate and interpretable, outperforming or matching state-of-the-art black-box methods.

Figure 2: Illustration of our construction of logical entity function and representation. (a): Each logical function is defined in a recursive manner with operations defined in Definition 1 (b): we build LERP in a feed-forward style, by constructing intermediate logical functions column by column.