[Paper Review] KAER: A Knowledge Augmented Pre-Trained Language Model for Entity Resolution

Liri Fang, Lan Li+3 more|arXiv (Cornell University)|Jan 12, 2023

Data Quality and Management4 citations

TL;DR

KAER proposes a knowledge-augmented pre-trained language model for entity resolution by injecting schema-level (column semantic types) and entity-level (Wikidata-linked entities) knowledge via prompting. It outperforms state-of-the-art models like Ditto, especially on dirty and textual datasets, with significant F1 improvements on structured and product domains, demonstrating robustness and effectiveness through knowledge injection and optimized prompting strategies.

ABSTRACT

Entity resolution has been an essential and well-studied task in data cleaning research for decades. Existing work has discussed the feasibility of utilizing pre-trained language models to perform entity resolution and achieved promising results. However, few works have discussed injecting domain knowledge to improve the performance of pre-trained language models on entity resolution tasks. In this study, we propose Knowledge Augmented Entity Resolution (KAER), a novel framework named for augmenting pre-trained language models with external knowledge for entity resolution. We discuss the results of utilizing different knowledge augmentation and prompting methods to improve entity resolution performance. Our model improves on Ditto, the existing state-of-the-art entity resolution method. In particular, 1) KAER performs more robustly and achieves better results on "dirty data", and 2) with more general knowledge injection, KAER outperforms the existing baseline models on the textual dataset and dataset from the online product domain. 3) KAER achieves competitive results on highly domain-specific datasets, such as citation datasets, requiring the injection of expert knowledge in future work.

Motivation & Objective

• To address the limitations of pre-trained language models in entity resolution when applied to heterogeneous, low-resource, or dirty data sources.
• To investigate how external domain knowledge—specifically column semantic types and linked entity types—can improve entity resolution performance.
• To evaluate the impact of different prompting techniques (space, slash, constrained tuning) on knowledge-augmented model performance.
• To assess the robustness and generalization of knowledge injection across diverse data types and domains, including citation, product, and music datasets.
• To identify scenarios where general knowledge injection is insufficient and where expert or domain-specific knowledge is required for optimal performance.

Proposed method

• Injects column semantic types (CST) by inferring fine-grained semantic types (e.g., ALBUM, ARTIST) from column values using NLP techniques.
• Performs entity linking (EL) by identifying entity mentions in text and linking them to Wikidata entries with associated semantic types.
• Applies three prompting strategies: space-separated, slash-separated, and constrained tuning to integrate injected knowledge into the input sequence.
• Uses RoBERTa as the base pre-trained language model and fine-tunes it on serialized entity pair inputs augmented with CST and EL signals.
• Employs a two-stage knowledge injection pipeline: first, column-level semantic type inference; second, entity mention detection and Wikidata linking.
• Evaluates model performance across multiple datasets with varying data quality (dirty vs. clean), data types (textual, structured), and domains (product, music, citation).

Figure 1: The framework of KAER. First, (a) column semantic type augmentation and (b) entity semantic type augmentation are used to inject domain knowledge from both column and entity levels of the input table. Then three options of prompting types are provided after external knowledge injection: (1

Experimental results

Research questions

• RQ1 How does injecting schema-level (column semantic type) and entity-level (Wikidata-linked) knowledge affect entity resolution performance in low-resource or dirty data settings?
• RQ2 Which prompting method (space, slash, constrained tuning) yields the best performance when integrating external knowledge into pre-trained language models for entity resolution?
• RQ3 Does knowledge augmentation improve model robustness on noisy or heterogeneous data sources compared to baseline models like RoBERTa and Ditto?
• RQ4 In what domains does general knowledge injection (e.g., from Wikidata) significantly improve performance, and where is it insufficient?
• RQ5 To what extent does knowledge injection degrade performance on datasets requiring expert or domain-specific knowledge, such as citation records?

Key findings

• KAER significantly outperforms RoBERTa and Ditto on the Structured/iTunes-Amazon dataset, achieving a 24.45% and 39.15% improvement in F1 score, respectively, with statistical significance.
• On the textual Abt-Buy dataset from the online product domain, KAER (RoBERTa+CST+/) achieves an F1 score of 92.61%, outperforming both baselines with statistical significance.
• On the Dirty/iTunes-Amazon dataset, KAER (RoBERTa+EL+/) achieves an F1 score of 81.82%, showing notable improvement over baselines, though not always statistically significant.
• The PCT prompting method improves robustness on low-quality data: KAER (RoBERTa+CST+PCT) achieves 72.00% F1 on Dirty/DBLP-GoogleScholar, outperforming KAER (RoBERTa+CST) at 62.96%.
• On the citation dataset Structured/DBLP-ACM, KAER (RoBERTa+CST+/) achieves the highest F1 score of 98.99%, significantly outperforming Ditto at the 95% confidence level.
• Knowledge injection fails to improve performance on datasets requiring expert knowledge (e.g., citation records), indicating a need for domain-specific knowledge injection in future work.