[Paper Review] ConE: Cone Embeddings for Multi-Hop Reasoning over Knowledge Graphs

Zhanqiu Zhang, Jie Wang+3 more|arXiv (Cornell University)|Oct 26, 2021

Advanced Graph Neural NetworksComputer Science33 references44 citations

TL;DR

ConE introduces cone-based query embeddings that support all FOL operations (conjunction, disjunction, negation) for multi-hop reasoning over knowledge graphs, achieving state-of-the-art results on standard benchmarks.

ABSTRACT

Query embedding (QE) -- which aims to embed entities and first-order logical (FOL) queries in low-dimensional spaces -- has shown great power in multi-hop reasoning over knowledge graphs. Recently, embedding entities and queries with geometric shapes becomes a promising direction, as geometric shapes can naturally represent answer sets of queries and logical relationships among them. However, existing geometry-based models have difficulty in modeling queries with negation, which significantly limits their applicability. To address this challenge, we propose a novel query embedding model, namely Cone Embeddings (ConE), which is the first geometry-based QE model that can handle all the FOL operations, including conjunction, disjunction, and negation. Specifically, ConE represents entities and queries as Cartesian products of two-dimensional cones, where the intersection and union of cones naturally model the conjunction and disjunction operations. By further noticing that the closure of complement of cones remains cones, we design geometric complement operators in the embedding space for the negation operations. Experiments demonstrate that ConE significantly outperforms existing state-of-the-art methods on benchmark datasets.

Motivation & Objective

• Motivate and enable robust multi-hop reasoning over incomplete KGs using first-order logic queries.
• Provide a geometry-based QE model capable of modeling all FOL operators, including negation.
• Represent queries and entities as Cartesian products of two-dimensional cones to naturally capture set operations.
• Develop projection, intersection, union, and complement operators in the embedding space and a learnable distance-based training objective.

Proposed method

• Represent entities and queries as Cartesian products of two-dimensional sector-cones; an entity is a cone with aperture 0, an answer set corresponds to cones contained within query cones.
• Define a projection operator f_r that translates query embeddings for a relation using an MLP-based parametric mapping on cone parameters.
• Model conjunction via intersection by computing a semantic-centered average of axis parameters and a cardinality-based minimum of apertures (CardMin).
• Model disjunction via a final union step using a set of cone embeddings corresponding to each disjunct (DNF formulation).
• Model negation via closure-complement on cones so that the complement of a cone remains a cone; derive theta_ax and theta_ap transformations for negation.
• Learn embeddings with a distance-based objective combining an outside and inside distance (d_con) and a Min over disjuncts (d_dis); apply negative sampling to train.

Experimental results

Research questions

• RQ1 Can a geometric, cone-based embedding model faithfully represent all FOL operations (existential quantification, conjunction, disjunction, negation) for KG question answering?
• RQ2 Do cone-based embeddings improve multi-hop reasoning performance and generalization, especially on queries involving negation, compared to state-of-the-art baselines?
• RQ3 How do aperture and axis parameters correlate with answer set cardinality and query semantics?
• RQ4 Is the Disjunctive Normal Form (DNF) formulation essential for handling unions effectively in geometry-based QE models?

Key findings

• ConE significantly outperforms state-of-the-art baselines on benchmarks without negation, with average improvements of 19.7% over BETA E across FB15k, FB237, and NELL.
• ConE shows up to 24.2% relative improvement over Query2Box on non-negation queries.
• For negation, ConE achieves higher MRR than BETA E across FB15k, FB237, and NELL (AVG gains of 14.8 vs 11.8 on FB15k, 5.9 vs 5.4 on FB237, and 6.4 vs 5.9 on NELL).
• ConE accommodates negation effectively (2in, 3in, inp, pin, pni, etc.) and achieves notable gains in negation-related metrics across datasets (average improvements).
• The model uses a semantic-average mechanism for intersection and a CardMin with DeepSets for aperture minimization, enabling scalable conjunction handling.
• Empirical results include detailed MRR table values across datasets and query structures demonstrating Consistent gains over baselines.