[Paper Review] Traversing Knowledge Graphs in Vector Space

Kelvin Guu, J. J. Miller+1 more|arXiv (Cornell University)|Jun 3, 2015

Topic Modeling24 references98 citations

TL;DR

This paper proposes a compositional training approach that enhances vector space models for knowledge base completion, enabling accurate path query answering despite missing facts. By recursively applying learned embeddings and introducing a novel compositional objective, the method reduces cascading errors and improves performance on both path queries (up to 2x accuracy gain) and standard knowledge base completion (up to 43% error reduction).

ABSTRACT

Path queries on a knowledge graph can be used to answer compositional questions such as "What languages are spoken by people living in Lisbon?". However, knowledge graphs often have missing facts (edges) which disrupts path queries. Recent models for knowledge base completion impute missing facts by embedding knowledge graphs in vector spaces. We show that these models can be recursively applied to answer path queries, but that they suffer from cascading errors. This motivates a new "compositional" training objective, which dramatically improves all models' ability to answer path queries, in some cases more than doubling accuracy. On a standard knowledge base completion task, we also demonstrate that compositional training acts as a novel form of structural regularization, reliably improving performance across all base models (reducing errors by up to 43%) and achieving new state-of-the-art results.

Motivation & Objective

• Address the challenge of answering compositional path queries on incomplete knowledge graphs where missing facts disrupt traversal.
• Overcome cascading errors in existing vector space models that arise when composing multiple relation embeddings for long paths.
• Develop a generalizable training objective that improves both path query performance and standard knowledge base completion.
• Demonstrate that compositional training acts as a form of structural regularization, enhancing generalization across diverse models.

Proposed method

• Interpret base knowledge base completion models (e.g., bilinear, TransE) as soft edge traversal operators in vector space.
• Recursively compose these operators to predict multi-hop paths by applying vector transformations (e.g., matrix multiplication) across relation embeddings.
• Introduce a compositional training objective that explicitly optimizes for path query accuracy by minimizing prediction error on composed paths.
• Use a differentiable path denotation representation where entity sets are modeled as low-dimensional vectors, enabling gradient-based optimization.
• Regularize the spatial arrangement of entity vectors by encouraging alignment between predicted path denotations and ground-truth sets.
• Train models on synthetic path queries derived from existing triples, augmenting the standard KBC dataset with compositional supervision.

Experimental results

Research questions

• RQ1 Can vector space models for knowledge base completion be effectively compositionalized to answer multi-hop path queries?
• RQ2 Does a dedicated compositional training objective reduce cascading errors in path query prediction compared to standard KBC training?
• RQ3 To what extent does compositional training improve performance on standard knowledge base completion tasks?
• RQ4 Can compositional training be viewed as a form of structural regularization that generalizes across different base models?
• RQ5 How does the model’s performance vary with path length and precision of intermediate relations?

Key findings

• Compositional training improves path query accuracy by more than doubling performance in some cases, particularly for paths of length 5.
• The method reduces error rates in standard knowledge base completion by up to 43% across all tested base models, achieving new state-of-the-art results.
• The compositional objective effectively regularizes vector space geometry, with higher precision paths showing greater reduction in embedding distance to target relations.
• Models trained with compositional objectives generalize better to unseen or hypothetical entities, such as Abraham Lincoln’s non-existent daughter.
• The compositional training objective leads to more accurate path denotations by aligning predicted sets with ground-truth sets through vector-space composition.
• The approach is broadly applicable to composable models such as bilinear models and TransE, demonstrating consistent gains across architectures.