[Paper Review] Low-resource Deep Entity Resolution with Transfer and Active Learning

Jungo Kasai, Kun Qian+3 more|arXiv (Cornell University)|Jun 17, 2019

Data Quality and ManagementDecision Sciences34 references17 citations

TL;DR

This paper proposes a deep learning-based entity resolution method that combines transfer learning and active learning to achieve state-of-the-art performance in low-resource settings. By pretraining on high-resource datasets and iteratively selecting informative examples via active learning, the model achieves comparable or better F1 scores using only 1/10th the labeled data compared to prior methods.

ABSTRACT

Entity resolution (ER) is the task of identifying different representations of the same real-world entities across databases. It is a key step for knowledge base creation and text mining. Recent adaptation of deep learning methods for ER mitigates the need for dataset-specific feature engineering by constructing distributed representations of entity records. While these methods achieve state-of-the-art performance over benchmark data, they require large amounts of labeled data, which are typically unavailable in realistic ER applications. In this paper, we develop a deep learning-based method that targets low-resource settings for ER through a novel combination of transfer learning and active learning. We design an architecture that allows us to learn a transferable model from a high-resource setting to a low-resource one. To further adapt to the target dataset, we incorporate active learning that carefully selects a few informative examples to fine-tune the transferred model. Empirical evaluation demonstrates that our method achieves comparable, if not better, performance compared to state-of-the-art learning-based methods while using an order of magnitude fewer labels.

Motivation & Objective

• Address the challenge of low-resource entity resolution where labeled data is scarce and costly to obtain.
• Overcome the data-hungry nature of deep learning models in entity resolution by leveraging transfer learning from high-resource source datasets.
• Improve model performance in low-resource target datasets through active learning that selects informative, uncertain, and high-confidence examples.
• Develop a unified deep learning architecture that learns transferable, attribute-agnostic representations across diverse ER scenarios.
• Demonstrate that combining transfer and active learning enables high-accuracy ER with minimal human-labeled data.

Proposed method

• Design a deep neural network architecture that learns transferable representations from multiple high-resource source datasets using domain adaptation.
• Apply transfer learning by initializing the model on source datasets with abundant labels and fine-tuning on the low-resource target dataset.
• Implement an active learning strategy that selects examples based on high-confidence predictions and uncertain predictions via a partition mechanism.
• Use a similarity vector for each attribute in an attribute-agnostic manner, preserving structural information across entity records.
• Integrate uncertainty sampling and confidence-based sampling to balance precision and recall during active learning iterations.
• Train the model end-to-end with a contrastive loss to optimize for match/non-match classification in low-resource settings.

Experimental results

Research questions

• RQ1 Can transfer learning from high-resource datasets improve deep learning-based entity resolution in low-resource target datasets?
• RQ2 How effective is active learning in reducing the number of required labeled examples for deep ER models?
• RQ3 Does combining transfer learning with active learning yield better performance than either method alone in low-resource ER?
• RQ4 Can a deep learning model learn attribute-agnostic, transferable representations that generalize across diverse ER domains?
• RQ5 How does the proposed method compare quantitatively to state-of-the-art learning-based and non-learning-based ER methods under label scarcity?

Key findings

• The proposed method achieves an F1 score of 97.73 with only 300 labeled examples on the DBLP-ACM dataset, outperforming baseline models with significantly more labels.
• Using high-confidence sampling and a partition mechanism in active learning improved recall by 8.2% (from 97.21% to 97.84%) while maintaining high precision (97.63%).
• The combination of transfer learning and active learning reduced the number of required labeled examples by an order of magnitude compared to state-of-the-art deep learning methods.
• The model achieved comparable or better performance than non-deep learning baselines (e.g., SVM, random forest) even with far fewer labeled examples.
• The partition mechanism successfully identified more false negatives, reducing the ratio of misclassified examples in the labeled set by 21.7% compared to baseline active learning.
• Empirical evaluation across multiple datasets (DBLP-Scholar, Fodors-Zagats, Zomato-Yelp, Cora) confirmed consistent performance gains in low-resource settings.