[Paper Review] Embedding Logical Queries on Knowledge Graphs

William L. Hamilton, Payal Bajaj+3 more|arXiv (Cornell University)|Jun 5, 2018

Advanced Graph Neural NetworksComputer Science62 citations

TL;DR

The paper introduces Graph Query Embedding (GQE), an embedding-based framework to answer conjunctive logical queries on incomplete knowledge graphs by learning geometric projections and intersections in a low-dimensional space, enabling efficient query evaluation.

ABSTRACT

Learning low-dimensional embeddings of knowledge graphs is a powerful approach used to predict unobserved or missing edges between entities. However, an open challenge in this area is developing techniques that can go beyond simple edge prediction and handle more complex logical queries, which might involve multiple unobserved edges, entities, and variables. For instance, given an incomplete biological knowledge graph, we might want to predict "em what drugs are likely to target proteins involved with both diseases X and Y?" -- a query that requires reasoning about all possible proteins that {\em might} interact with diseases X and Y. Here we introduce a framework to efficiently make predictions about conjunctive logical queries -- a flexible but tractable subset of first-order logic -- on incomplete knowledge graphs. In our approach, we embed graph nodes in a low-dimensional space and represent logical operators as learned geometric operations (e.g., translation, rotation) in this embedding space. By performing logical operations within a low-dimensional embedding space, our approach achieves a time complexity that is linear in the number of query variables, compared to the exponential complexity required by a naive enumeration-based approach. We demonstrate the utility of this framework in two application studies on real-world datasets with millions of relations: predicting logical relationships in a network of drug-gene-disease interactions and in a graph-based representation of social interactions derived from a popular web forum.

Motivation & Objective

• Motivate answering complex conjunctive queries beyond single-edge prediction on incomplete KGs.
• Develop a scalable embedding-based framework that maps conjunctive queries to low-dimensional embeddings.
• Demonstrate that geometric operations can approximate query denotations and support efficient inference.
• Evaluate on large real-world datasets (biomedical drug–gene–disease network and Reddit interactions).
• Show that training on complex queries improves performance over edge-only training.

Proposed method

• Embed graph nodes in a d-dimensional space with trainable node embeddings.
• Represent logical edges with a geometric projection operator P that maps q to P(q, tau) = R_tau q.
• Introduce a geometric intersection operator I that aggregates multiple query embeddings into an intersection embedding.
• Use Algorithm 1 to compute a query embedding by DAG-structured propagation of P and I across anchor nodes and variables.
• Score node v for a query q via cosine similarity score(q, z_v).
• Train P, I, and node embeddings with a max-margin loss using positive and negative examples; employ hard negatives for intersection queries.
• Perform efficient inference via nearest-neighbor search (e.g., locality-sensitive hashing) in the embedding space.

Experimental results

Research questions

• RQ1 Can conjunctive graph queries over incomplete knowledge graphs be answered efficiently via embeddings?
• RQ2 Can a small set of learned geometric operators faithfully represent existentially quantified query denotations?
• RQ3 How does GQE compare to enumeration-based edge-prediction baselines on complex queries?
• RQ4 Does training on complex queries improve downstream performance beyond edge-level training?
• RQ5 What is the scalability of GQE on million-edge real-world datasets?

Key findings

• GQE achieves strong predictive performance on Bio and Reddit datasets, with Bilinear GQE performing best (Bio AUC 91.0; Reddit AUC 76.4).
• GQE outperforms an enumeration baseline in the restricted no-bound-variables setting.
• Training on complex queries substantially improves AUC compared to edge-training alone (average ~13% across datasets, p<0.001).
• Query embeddings enable linear-time complexity in the number of query edges, with sublinear inference via nearest-neighbor search.
• The framework supports variants using DistMult and TransE projections, with Bilinear often yielding the best results among tested configurations.