[Paper Review] RDF2Vec Light -- A Lightweight Approach for Knowledge Graph Embeddings

Jan Portisch, Michael Hladik+1 more|arXiv (Cornell University)|Sep 16, 2020

Advanced Graph Neural Networks7 references4 citations

TL;DR

RDF2Vec Light is a lightweight knowledge graph embedding method that generates dense vector representations for only a subset of entities of interest, rather than the entire knowledge graph. By performing localized random walks around target entities and training embeddings only on these subgraphs, it achieves performance comparable to full-graph RDF2Vec while reducing runtime and hardware requirements by orders of magnitude.

ABSTRACT

Knowledge graph embedding approaches represent nodes and edges of graphs as mathematical vectors. Current approaches focus on embedding complete knowledge graphs, i.e. all nodes and edges. This leads to very high computational requirements on large graphs such as DBpedia or Wikidata. However, for most downstream application scenarios, only a small subset of concepts is of actual interest. In this paper, we present RDF2Vec Light, a lightweight embedding approach based on RDF2Vec which generates vectors for only a subset of entities. To that end, RDF2Vec Light only traverses and processes a subgraph of the knowledge graph. Our method allows the application of embeddings of very large knowledge graphs in scenarios where such embeddings were not possible before due to a significantly lower runtime and significantly reduced hardware requirements.

Motivation & Objective

• To address the high computational cost and memory overhead of training full knowledge graph embeddings on large-scale graphs like DBpedia and Wikidata.
• To enable downstream applications—such as classification, regression, and relatedness tasks—to use knowledge graph embeddings in resource-constrained environments.
• To develop a method that generates high-quality embeddings for a small, task-specific set of entities without retraining on the entire graph.
• To reduce training time and model size while maintaining competitive performance compared to classic RDF2Vec.

Proposed method

• The method generates random walks only around entities of interest, using a bidirectional walk strategy that alternates between predecessor and successor nodes at each step.
• Walks are initiated from each entity of interest and expanded to a fixed depth, with traversal direction chosen randomly at each step between incoming and outgoing edges.
• The walk generation process is confined to a subgraph induced by the entities of interest and their reachable neighbors, avoiding traversal of the entire knowledge graph.
• After walk generation, standard skip-gram or CBOW training is applied to the generated walks to learn dense vector representations.
• The approach supports multiple RDF formats (Turtle, n-triples, RDF/XML, HDT) and includes a REST API for deployment.
• The implementation is available in Java and Python, with support for both CPU and GPU inference.

Experimental results

Research questions

• RQ1 Can knowledge graph embeddings be effectively learned for only a subset of entities, rather than the entire graph, without significant performance loss?
• RQ2 How does the performance of a localized embedding method compare to full-graph RDF2Vec on downstream tasks like classification and regression?
• RQ3 What is the impact of entity connectivity (linkage degree) and subgraph homogeneity on the performance of localized embeddings?
• RQ4 To what extent can computational cost and model size be reduced by training only on relevant subgraphs?
• RQ5 How does the bidirectional walk strategy improve context representation for entities of interest compared to unidirectional walks?

Key findings

• RDF2Vec Light achieves performance comparable to classic RDF2Vec on classification and regression tasks, with only a fraction of the training time and memory usage.
• For the document and entity relatedness tasks, RDF2Vec Light performs comparably to classic RDF2Vec in the CBOW variant, but underperforms in the SG variant, particularly on dense or heterogeneous subgraphs.
• The performance of RDF2Vec Light correlates strongly with subgraph density: higher density (fewer head entities, more homogeneous sets) leads to better results.
• Runtime scales linearly with the number of entities of interest, requiring approximately 1 minute per 10 entities on commodity hardware, compared to several days for full-graph training.
• Model sizes are drastically reduced—typically just a few kilobytes—compared to classic models that require multiple gigabytes of disk space.
• The method enables practical deployment of knowledge graph embeddings in low-resource environments where full-graph embeddings were previously infeasible.