[Paper Review] Symbolic Querying of Vector Spaces: Probabilistic Databases Meets Relational Embeddings

Tal Friedman, Guy Van den Broeck|arXiv (Cornell University)|Feb 24, 2020

Bayesian Modeling and Causal Inference39 references4 citations

TL;DR

This paper introduces TractOR, a novel relational embedding model that unifies probabilistic databases (PDBs) with relational embeddings to enable efficient, principled probabilistic querying of incomplete and uncertain data. By embedding entities and relations as vectors and enforcing a factorization structure compatible with PDB semantics, TractOR supports tractable inference for complex queries while maintaining strong probabilistic foundations, outperforming baselines in query prediction and demonstrating robustness even under constraints like non-negative parameters.

ABSTRACT

We propose unifying techniques from probabilistic databases and relational embedding models with the goal of performing complex queries on incomplete and uncertain data. We formalize a probabilistic database model with respect to which all queries are done. This allows us to leverage the rich literature of theory and algorithms from probabilistic databases for solving problems. While this formalization can be used with any relational embedding model, the lack of a well-defined joint probability distribution causes simple query problems to become provably hard. With this in mind, we introduce \TO, a relational embedding model designed to be a tractable probabilistic database, by exploiting typical embedding assumptions within the probabilistic framework. Using a principled, efficient inference algorithm that can be derived from its definition, we empirically demonstrate that \TOs is an effective and general model for these querying tasks.

Motivation & Objective

• To address the limitations of existing relational embedding models, which lack consistent probabilistic semantics and struggle with complex querying beyond link prediction.
• To overcome the brittleness of probabilistic databases in populating meaningful probabilities from incomplete data and in handling complex queries efficiently.
• To unify the predictive power of relational embeddings with the formal probabilistic semantics of PDBs, enabling sound and tractable reasoning over uncertain, incomplete relational data.
• To design a model that supports not only link prediction but also complex query evaluation with guaranteed tractability and principled inference.

Proposed method

• Formalize a probabilistic database (PDB) framework as the underlying semantics for relational embeddings, ensuring well-defined independence assumptions and probabilistic consistency.
• Introduce TractOR, a relational embedding model designed to be a tractable PDB by embedding entities and relations as vectors and using a scoring function that respects the factorization structure of PDBs.
• Leverage the factorization assumption common in embedding models (e.g., DistMult) to ensure that the joint probability distribution is tractable and efficiently computable.
• Derive an efficient inference algorithm from the model’s definition, enabling fast and exact computation of query probabilities over the full probability space.
• Train TractOR using standard embedding techniques (negative sampling, max-margin loss), while preserving compatibility with PDB semantics for downstream reasoning.
• Evaluate TractOR on complex query prediction tasks, comparing against baselines including DistMult and TransE, using AUC and other metrics.

Experimental results

Research questions

• RQ1 Can a relational embedding model be designed to support not only link prediction but also complex probabilistic queries with tractable inference?
• RQ2 How can the probabilistic semantics of probabilistic databases be effectively combined with the predictive power of relational embeddings?
• RQ3 What structural assumptions in embedding models can be exploited to ensure tractability in probabilistic reasoning over uncertain data?
• RQ4 To what extent does enforcing a PDB-compatible factorization improve the efficiency and reliability of query answering compared to standard embedding models?

Key findings

• TractOR achieves competitive performance on query prediction tasks, outperforming or matching baselines such as DistMult and TransE, especially in complex query settings.
• The model maintains strong performance even when parameters are constrained to be non-negative, though performance degrades slightly compared to TractOR+, indicating that negative embeddings improve expressiveness and training stability.
• Empirical results show that TractOR is effective not only for link prediction but also for complex queries, as its ability to represent a full probability distribution enables efficient and accurate inference.
• The gap between TractOR and TractOR+ highlights the importance of model expressiveness: allowing negative values significantly improves link prediction, which in turn enhances query prediction performance.
• TractOR’s design enables efficient inference due to its factorization structure, making it possible to perform complex probabilistic reasoning that is provably hard in standard PDBs.