[Paper Review] EAGER: Embedding-Assisted Entity Resolution for Knowledge Graphs

Daniel Obraczka, Jonathan Schuchart+1 more|arXiv (Cornell University)|Jan 15, 2021

Data Quality and Management39 references4 citations

TL;DR

EAGER is a novel, supervised entity resolution system for knowledge graphs that combines graph embeddings and attribute similarities using machine learning classifiers. It outperforms state-of-the-art methods, especially on deeper knowledge graphs, by leveraging both structural and attribute-based features in a unified learning framework with statistically significant gains.

ABSTRACT

Entity Resolution (ER) is a constitutional part for integrating different knowledge graphs in order to identify entities referring to the same real-world object. A promising approach is the use of graph embeddings for ER in order to determine the similarity of entities based on the similarity of their graph neighborhood. The similarity computations for such embeddings translates to calculating the distance between them in the embedding space which is comparatively simple. However, previous work has shown that the use of graph embeddings alone is not sufficient to achieve high ER quality. We therefore propose a more comprehensive ER approach for knowledge graphs called EAGER (Embedding-Assisted Knowledge Graph Entity Resolution) to flexibly utilize both the similarity of graph embeddings and attribute values within a supervised machine learning approach. We evaluate our approach on 23 benchmark datasets with differently sized and structured knowledge graphs and use hypothesis tests to ensure statistical significance of our results. Furthermore we compare our approach with state-of-the-art ER solutions, where our approach yields competitive results for table-oriented ER problems and shallow knowledge graphs but much better results for deeper knowledge graphs.

Motivation & Objective

• To address the challenge of entity resolution in heterogeneous knowledge graphs with diverse entity types, schema mismatches, and varying attribute representations.
• To overcome the limitations of existing ER methods that rely heavily on schema matching or perform poorly on complex, deep knowledge graphs.
• To develop a generic, schema-agnostic ER system that effectively integrates both structural (graph embedding) and attribute-based similarity signals.
• To evaluate the effectiveness and efficiency of combining embeddings and attributes across diverse, real-world knowledge graph datasets.
• To establish a benchmark for entity resolution in knowledge graphs using 23 datasets of varying size and structure, including new movie domain data.

Proposed method

• EAGER employs graph embeddings—specifically those performing best in prior work (e.g., TransE, node2vec, DeepWalk)—to encode entities into low-dimensional vector spaces capturing topological and semantic relationships.
• It extracts attribute similarities by normalizing and comparing values across entities, including string, date, and numeric attributes using standard similarity functions.
• The system combines both embedding-based and attribute-based features into a single feature vector for each entity pair.
• A supervised machine learning classifier (e.g., MLP, Random Forest, XGBoost) is trained on labeled entity pairs to predict match or non-match.
• The framework supports flexible configuration, allowing plug-in of different embeddings and classifiers to evaluate performance trade-offs.
• Statistical hypothesis testing (Friedman and Nemenyi tests) is used to validate the significance of performance differences across methods.

Experimental results

Research questions

• RQ1 Can combining graph embeddings and attribute similarities in a supervised learning framework significantly improve entity resolution accuracy on heterogeneous knowledge graphs?
• RQ2 How does EAGER perform compared to state-of-the-art ER systems like Magellan and DeepMatcher across diverse knowledge graph structures?
• RQ3 Does the integration of embeddings and attributes yield better results than using either signal alone, particularly on deeper and more complex knowledge graphs?
• RQ4 Which combination of graph embeddings and classifiers yields the most robust and effective performance across different datasets?
• RQ5 Is the performance gain of EAGER statistically significant across multiple benchmark datasets?

Key findings

• EAGER significantly outperforms Magellan across all 23 benchmark datasets, with a mean rank of 1.125 in the Nemenyi test, indicating superior overall performance.
• On deeper knowledge graphs, EAGER with an MLP classifier achieves significantly better F-measure than DeepMatcher, with a p-value of 2.21×10⁻¹¹ from the Friedman test, confirming statistical significance.
• For table-oriented and shallow knowledge graphs, EAGER performs competitively with state-of-the-art methods, achieving F-measures above 0.98 on several datasets.
• The combination of graph embeddings and attribute similarities consistently yields higher F-measure than using either signal in isolation, demonstrating the value of multimodal feature fusion.
• On the 100K-sized datasets, EAGER’s MLP model achieved an F-measure of 0.995 on D-Y(V2), outperforming DeepMatcher (0.993) and Magellan (0.984).
• The critical distance diagram from the Nemenyi test confirms that EAGER (MLP) is significantly better than both DeepMatcher and Magellan, with no overlapping confidence intervals.