[Paper Review] Entity Alignment For Knowledge Graphs: Progress, Challenges, and Empirical Studies

Deepak Chaurasiya, Anil Surisetty+6 more|arXiv (Cornell University)|May 18, 2022

Data Quality and Management4 citations

TL;DR

This paper presents a comprehensive survey and empirical analysis of embedding-based entity alignment (EA) methods for knowledge graphs, introducing four research questions focused on hubness, degree distribution, non-isomorphic neighborhoods, and name bias. It proposes a novel graph sampling method to generate a low-name-bias dataset, introduces an h-score to quantify hubness, and releases an open-source library with 14 EA methods, enabling robust benchmarking and improved generalization in industrial settings.

ABSTRACT

Entity Alignment (EA) identifies entities across databases that refer to the same entity. Knowledge graph-based embedding methods have recently dominated EA techniques. Such methods map entities to a low-dimension space and align them based on their similarities. With the corpus of EA methodologies growing rapidly, this paper presents a comprehensive analysis of various existing EA methods, elaborating their applications and limitations. Further, we distinguish the methods based on their underlying algorithms and the information they incorporate to learn entity representations. Based on challenges in industrial datasets, we bring forward $4$ research questions (RQs). These RQs empirically analyse the algorithms from the perspective of extit{Hubness, Degree distribution, Non-isomorphic neighbourhood,} and extit{Name bias}. For Hubness, where one entity turns up as the nearest neighbour of many other entities, we define an $h$-score to quantify its effect on the performance of various algorithms. Additionally, we try to level the playing field for algorithms that rely primarily on name-bias existing in the benchmarking open-source datasets by creating a low name bias dataset. We further create an open-source repository for $14$ embedding-based EA methods and present the analysis for invoking further research motivations in the field of EA.

Motivation & Objective

• To address the lack of robust evaluation frameworks for entity alignment (EA) methods in industrial settings, where real-world data often contains biases and structural challenges.
• To investigate how common EA methods perform under real-world data issues such as hubness, degree variance, non-isomorphic subgraphs, and name bias.
• To develop a novel graph sampling technique that preserves graph properties while minimizing name bias for more reliable benchmarking.
• To release an open-source library implementing 14 embedding-based EA methods, ensuring reproducibility and fostering further research.
• To provide empirical insights into which methods are most resilient to structural and linguistic biases in real-world knowledge graphs.

Proposed method

• Proposes a new categorization of EA methods based on the type of network information they exploit: structure-aware (S), structure and relation-aware (SR), and structure and attribute-aware (SA).
• Introduces an h-score metric to quantify the extent of hubness in alignment results, where a high h-score indicates that a few entities are repeatedly selected as nearest neighbors.
• Develops a novel graph sampling technique that selects entity pairs with low name similarity while preserving global graph properties such as degree distribution and clustering coefficient.
• Constructs a cross-lingual, low-name-bias benchmark dataset (EN-FR LNB) by applying the sampling method to existing datasets, enabling fairer evaluation of EA algorithms.
• Employs a framework where entity embeddings are learned via knowledge graph embedding models (e.g., TransE, GNNs), followed by nearest neighbor search in the embedding space for alignment.
• Uses BERT-based embeddings for name similarity evaluation and compares performance across datasets with varying name bias levels to isolate linguistic dependence.

Experimental results

Research questions

• RQ1 Does the method address the hubness problem present in knowledge graphs, where a few entities become nearest neighbors to many others?
• RQ2 Does the method maintain alignment accuracy across entities of varying degrees, especially in sparse or highly connected regions of the graph?
• RQ3 Can the method align entities that have non-isomorphic subgraphs across two knowledge graphs due to source differences or incompleteness?
• RQ4 Does name bias in entity names lead to inflated accuracy scores, and how does the method perform on low-name-bias datasets?

Key findings

• The h-score metric reveals that hubness significantly impacts alignment performance, with methods like MTransE showing higher sensitivity to hub entities.
• Alignment accuracy drops sharply for low-degree entities in structure-only methods, while methods using attributes (e.g., AttrE) or linguistic features (e.g., RDGCN) show more consistent performance.
• RDGCN and HGCN achieve high accuracy on the high-bias EN-FR V1 dataset (75.17 and 77.93 Hits@1, respectively), but their performance drops significantly on the low-bias EN-FR LNB dataset (61.10 and 65.63 Hits@1), indicating strong name bias dependence.
• AttrE shows the most robust performance across datasets, with a smaller drop in accuracy (from 45.25 to 38.77 Hits@1) when name bias is reduced, demonstrating its reliance on multi-attribute information.
• Methods relying solely on structural information (e.g., MTransE) show a marked decline in alignment accuracy when dealing with non-isomorphic neighborhoods, while those using attribute or linguistic features (e.g., AliNet, RDGCN) maintain better performance.
• The proposed low-name-bias dataset (EN-FR LNB) reduces BERT-based name similarity to 0.57, exposing the true generalization capability of EA models and revealing that name-based methods are not reliable in real-world, low-similarity scenarios.