[Paper Review] Query2box: Reasoning over Knowledge Graphs in Vector Space using Box Embeddings

Hongyu Ren, Weihua Hu+1 more|arXiv (Cornell University)|Feb 14, 2020

Advanced Graph Neural NetworksComputer Science33 references81 citations

TL;DR

Query2box embeds queries as boxes in vector space to answer existential positive first-order (EPFO) queries over large, incomplete KGs, using DNFs to handle disjunctions and achieving up to 25% relative improvement over baselines.

ABSTRACT

Answering complex logical queries on large-scale incomplete knowledge graphs (KGs) is a fundamental yet challenging task. Recently, a promising approach to this problem has been to embed KG entities as well as the query into a vector space such that entities that answer the query are embedded close to the query. However, prior work models queries as single points in the vector space, which is problematic because a complex query represents a potentially large set of its answer entities, but it is unclear how such a set can be represented as a single point. Furthermore, prior work can only handle queries that use conjunctions ($\\wedge$) and existential quantifiers ($\\exists$). Handling queries with logical disjunctions ($\\vee$) remains an open problem. Here we propose query2box, an embedding-based framework for reasoning over arbitrary queries with $\\wedge$, $\\vee$, and $\\exists$ operators in massive and incomplete KGs. Our main insight is that queries can be embedded as boxes (i.e., hyper-rectangles), where a set of points inside the box corresponds to a set of answer entities of the query. We show that conjunctions can be naturally represented as intersections of boxes and also prove a negative result that handling disjunctions would require embedding with dimension proportional to the number of KG entities. However, we show that by transforming queries into a Disjunctive Normal Form, query2box is capable of handling arbitrary logical queries with $\\wedge$, $\\vee$, $\\exists$ in a scalable manner. We demonstrate the effectiveness of query2box on three large KGs and show that query2box achieves up to 25% relative improvement over the state of the art.

Motivation & Objective

• Motivate the need to answer complex, incomplete KG queries beyond conjunctive queries.
• Introduce box embeddings to represent query denotations as regions that enclose answer entities.
• Develop a DNF-based approach to tractably handle disjunctions in EPFO queries.
• Define geometric operators (projection and intersection) on boxes and a distance-based training objective.
• Demonstrate scalability and improved accuracy on large KB benchmarks compared to state-of-the-art baselines.

Proposed method

• Embed KG entities as vectors and queries as axis-aligned boxes in R^d.
• Define projection as a translation and enlargement of a box: Box_p + r for relation r.
• Define intersection as a learnable, attention-guided shrinkage of boxes to model set intersection.
• Model distance dist_box(v; q) with a weighted blend of outside and inside box distances to rank entities.
• Train via negative sampling using a margin-based loss encouraging true answers to be closer than negatives.
• Transform EPFO queries to DNFs to handle unions by solving multiple conjunctive queries and aggregating results with a min-distance rule.
• Aggregate distances across conjunctive queries: dist_agg(v; q) = min(dist_box(v; q^(i))) over all CNF components.
• Provide scalability via constant-time box operations and parallelizable evaluation, with NN-based nearest-neighbor search for retrieval.

Experimental results

Research questions

• RQ1 Can complex EPFO queries be represented and reasoned over in a low-dimensional vector space using boxes?
• RQ2 How can disjunctions be tractably handled within box embeddings without exploding dimensionality?
• RQ3 Does transforming EPFO queries to DNFs enable accurate and scalable query answering on large, incomplete KGs?
• RQ4 What accuracy gains and generalization capabilities does query2box provide compared to state-of-the-art baselines on standard KG benchmarks?

Key findings

• Query2box achieves up to 25% relative improvement over strong baselines on EPFO query answering tasks.
• Box embeddings naturally model sets of answers and allow closed-form composition under conjunction and disjunction via DNFs.
• Transforming EPFO queries to DNFs enables tractable reasoning in low-dimensional space while preserving expressive power.
• Query2box demonstrates strong generalization to unseen query structures and implicit handling of missing relations.
• Experiments on FB15k, FB15k-237, and NELL995 show better performance for complex query structures (e.g., 2p, 3p, 2i, 3i, ip, pi, 2u, up) compared to baselines.