[Paper Review] Open-World Knowledge Graph Completion

Baoxu Shi, Tim Weninger|arXiv (Cornell University)|Nov 9, 2017

Topic Modeling26 references14 citations

TL;DR

This paper introduces ConMask, a novel open-world knowledge graph completion model that uses relationship-dependent content masking and a fully convolutional neural network to extract relevant textual features from entity descriptions, enabling accurate prediction of missing relationships involving unseen or sparsely connected entities. ConMask outperforms existing models on both open-world and closed-world KGC benchmarks, demonstrating strong generalization and robustness on large-scale datasets like DBPedia50k and DBPedia500k.

ABSTRACT

Knowledge Graphs (KGs) have been applied to many tasks including Web search, link prediction, recommendation, natural language processing, and entity linking. However, most KGs are far from complete and are growing at a rapid pace. To address these problems, Knowledge Graph Completion (KGC) has been proposed to improve KGs by filling in its missing connections. Unlike existing methods which hold a closed-world assumption, i.e., where KGs are fixed and new entities cannot be easily added, in the present work we relax this assumption and propose a new open-world KGC task. As a first attempt to solve this task we introduce an open-world KGC model called ConMask. This model learns embeddings of the entity's name and parts of its text-description to connect unseen entities to the KG. To mitigate the presence of noisy text descriptions, ConMask uses a relationship-dependent content masking to extract relevant snippets and then trains a fully convolutional neural network to fuse the extracted snippets with entities in the KG. Experiments on large data sets, both old and new, show that ConMask performs well in the open-world KGC task and even outperforms existing KGC models on the standard closed-world KGC task.

Motivation & Objective

• To address the limitations of closed-world knowledge graph completion (KGC) models that cannot predict relationships for unseen or poorly connected entities.
• To relax the closed-world assumption by introducing an open-world KGC task that allows prediction of triples involving new entities not present in the original KG.
• To develop a model that effectively uses textual descriptions as a substitute for topological features in low-connectivity or unseen entities.
• To improve KGC performance on sparse and evolving knowledge graphs by integrating semantic information from entity descriptions.

Proposed method

• ConMask uses relationship-dependent content masking to extract relevant textual snippets from entity descriptions based on the target relationship.
• It employs a fully convolutional neural network (FCNN) to fuse the masked textual snippets with entity embeddings for relationship prediction.
• The model combines semantic averaging of entity descriptions with learned representations from the FCNN to generate context-aware embeddings.
• It leverages textual features as a proxy for topological structure, enabling prediction for entities with few or no existing connections.
• The model is trained end-to-end using a margin-based ranking loss to optimize for correct tail entity prediction.
• ConMask is evaluated on both open-world and closed-world settings, including newly released DBPedia50k and DBPedia500k datasets.

Experimental results

Research questions

• RQ1 Can a KGC model effectively predict missing relationships for entities not present in the original knowledge graph, under an open-world assumption?
• RQ2 How well can textual descriptions compensate for the lack of topological connectivity in predicting new or missing relationships?
• RQ3 Does relationship-dependent content masking improve the relevance and performance of textual feature extraction in KGC?
• RQ4 Can a model designed for open-world KGC also perform competitively on standard closed-world KGC benchmarks?
• RQ5 How does ConMask compare to existing KGC models in terms of mean rank and MRR on large-scale, real-world knowledge graphs?

Key findings

• ConMask improves mean rank performance by at least 60% on both DBPedia50k and DBPedia500k datasets compared to a baseline semantic averaging model without content extraction.
• The performance gain from content extraction is more significant on the larger DBPedia500k dataset, indicating better scalability and utility of textual features in high-coverage settings.
• ConMask outperforms standard closed-world KGC models like TransE and TransR on the DBPedia50k dataset, particularly benefiting from its ability to handle sparse graphs.
• On the DBPedia500k dataset, ConMask achieves competitive results despite the absence of TransR results due to training failure, suggesting robustness and efficiency.
• The model successfully ranks correct entities within the top-3 predictions in most cases, including complex relationships like 'notable work' and 'writer', even when entity names do not appear directly in descriptions.
• ConMask exhibits a notable limitation in ranking entities that are semantically similar to the relationship (e.g., 'writer' for 'writer' relationship), which can degrade performance and requires future filtering.