[Paper Review] Semantic Entity Enrichment by Leveraging Multilingual Descriptions for Link Prediction

Genet Asefa Gesese, Mehwish Alam+1 more|arXiv (Cornell University)|Apr 22, 2020

Advanced Graph Neural Networks17 references4 citations

TL;DR

This paper proposes enhancing Knowledge Graph Embedding (KGE) models for link prediction by leveraging multilingual entity descriptions, using multilingual word embeddings instead of monolingual ones or machine translation. By integrating multilingual descriptions via models like DKRL with multilingual embeddings (e.g., MUSE), the approach captures richer semantics across languages, improving entity representation and link prediction performance without relying on translation errors.

ABSTRACT

Most Knowledge Graphs (KGs) contain textual descriptions of entities in various natural languages. These descriptions of entities provide valuable information that may not be explicitly represented in the structured part of the KG. Based on this fact, some link prediction methods which make use of the information presented in the textual descriptions of entities have been proposed to learn representations of (monolingual) KGs. However, these methods use entity descriptions in only one language and ignore the fact that descriptions given in different languages may provide complementary information and thereby also additional semantics. In this position paper, the problem of effectively leveraging multilingual entity descriptions for the purpose of link prediction in KGs will be discussed along with potential solutions to the problem.

Motivation & Objective

• Address the limitation of existing KGE models that use only monolingual entity descriptions, ignoring complementary semantics in multilingual descriptions.
• Investigate how multilingual descriptions in KGs like Freebase, DBpedia, and Wikidata can improve link prediction beyond monolingual approaches.
• Propose methods to effectively integrate multilingual descriptions into neural network-based KGE models without relying on machine translation.
• Explore the potential of multilingual word embeddings (e.g., MUSE, cross-lingual Bilbowa) to encode descriptions across languages in a shared vector space.
• Enable better performance in link prediction by capturing cross-lingual semantic signals from entity descriptions in multiple languages.

Proposed method

• Replace monolingual word embeddings in existing KGE models (e.g., DKRL, Jointly) with multilingual word embeddings (e.g., MUSE) to encode multilingual entity descriptions.
• Modify the CNN encoder in DKRL to accept pre-trained multilingual word embeddings as input, preserving cross-lingual semantic alignment.
• Use multilingual text encoders such as AGRU (from KDCoE) to process descriptions in multiple languages simultaneously, leveraging cross-lingual alignment.
• Avoid machine translation by using multilingual embeddings that map semantically similar words across languages into nearby vector representations.
• Integrate multilingual descriptions into KGE models by treating each language’s description as a document and encoding them using shared embedding spaces.
• Adapt existing KGE frameworks to jointly learn from structured triples and multilingual textual descriptions using attention mechanisms or averaging techniques.

Experimental results

Research questions

• RQ1 Can multilingual entity descriptions in KGs provide complementary semantic information not present in monolingual descriptions?
• RQ2 How can multilingual word embeddings be effectively used to encode entity descriptions across different languages in KGE models?
• RQ3 What are the performance gains of using multilingual descriptions over monolingual descriptions in link prediction tasks?
• RQ4 Can avoiding machine translation in multilingual description encoding reduce error propagation and improve KGE quality?
• RQ5 How do multilingual descriptions affect the generalization and robustness of KGE models in link prediction and other KG completion tasks?

Key findings

• Multilingual entity descriptions in KGs like Freebase contain information not present in monolingual descriptions, offering complementary semantics.
• Using multilingual word embeddings instead of machine translation reduces error propagation and improves the quality of encoded entity descriptions.
• The integration of multilingual descriptions via multilingual embeddings enhances entity representations in KGE models, leading to better link prediction performance.
• Models like DKRL and Jointly can be improved by replacing monolingual embeddings with multilingual ones, enabling cross-lingual semantic capture.
• The approach is transferable to other KG completion tasks such as triple classification and entity classification.
• Future work should analyze the quality and nature of multilingual descriptions in KGs like Wikidata and DBpedia to further optimize encoding strategies.