[Paper Review] A Pre-trained Data Deduplication Model based on Active Learning

Shi, Haochen, Xinyao Liu+5 more|arXiv (Cornell University)|Jul 31, 2023

Data Quality and Management4 citations

TL;DR

This paper proposes a pre-trained data deduplication model (PDDM-AL) that leverages active learning and R-Drop data augmentation to improve semantic-level duplicate detection. Built on a fine-tuned Transformer, it selects uncertain samples for labeling, reducing annotation cost while achieving a 28% recall improvement over SOTA on benchmark datasets.

ABSTRACT

In the era of big data, the issue of data quality has become increasingly prominent. One of the main challenges is the problem of duplicate data, which can arise from repeated entry or the merging of multiple data sources. These "dirty data" problems can significantly limit the effective application of big data. To address the issue of data deduplication, we propose a pre-trained deduplication model based on active learning, which is the first work that utilizes active learning to address the problem of deduplication at the semantic level. The model is built on a pre-trained Transformer and fine-tuned to solve the deduplication problem as a sequence to classification task, which firstly integrate the transformer with active learning into an end-to-end architecture to select the most valuable data for deduplication model training, and also firstly employ the R-Drop method to perform data augmentation on each round of labeled data, which can reduce the cost of manual labeling and improve the model's performance. Experimental results demonstrate that our proposed model outperforms previous state-of-the-art (SOTA) for deduplicated data identification, achieving up to a 28% improvement in Recall score on benchmark datasets.

Motivation & Objective

• To address the challenge of semantic-level data deduplication in big data environments where syntactic similarity fails to capture true duplicates.
• To reduce the high cost of manual labeling in training deep learning models for deduplication by integrating active learning.
• To enhance model robustness and performance using R-Drop data augmentation to stabilize training on limited labeled data.
• To establish a novel end-to-end framework combining pre-trained Transformers, active learning, and data augmentation for deduplication.

Proposed method

• Fine-tunes a pre-trained Transformer model to treat deduplication as a sequence-to-classification task, enabling contextual and semantic understanding of data pairs.
• Applies active learning with uncertainty sampling to iteratively select the most informative unlabeled samples for human annotation, minimizing labeling effort.
• Integrates R-Drop during each active learning round to regularize the model and improve generalization by encouraging consistency in predictions for augmented data.
• Injects domain-specific knowledge by marking key attributes (e.g., book edition) to improve attention to critical semantic features.
• Uses a selector-learner loop: the model selects uncertain samples, experts label them, and the model is retrained in an iterative, adaptive manner.
• Employs a dual-branch architecture to encode paired records and computes similarity scores for duplicate classification.

Experimental results

Research questions

• RQ1 Can active learning significantly reduce the number of labeled samples required for training a high-performing data deduplication model?
• RQ2 How does uncertainty-based sampling compare to random sampling in selecting informative data for deduplication?
• RQ3 To what extent does R-Drop improve model robustness and performance under limited labeled data?
• RQ4 Can a pre-trained Transformer model effectively capture semantic relationships in structured data for deduplication?
• RQ5 Does injecting domain-specific attribute markers enhance the model’s ability to detect semantic duplicates?

Key findings

• The uncertainty-based active learning strategy achieved a 28% higher recall score compared to state-of-the-art models on benchmark datasets.
• On the MusicBrainz-20-A01 dataset, the model reached a recall of 0.992 and F1 of 0.974 after 5,000 labeled pairs using uncertainty sampling.
• The uncertainty selection strategy consistently outperformed random selection, with a 3% higher F1 score on the GeographicSettlements dataset after three active learning rounds.
• R-Drop improved model stability and generalization, enabling consistent performance gains even with noisy or ambiguous data.
• The integration of domain-aware attribute marking enhanced the model’s focus on critical semantic features, such as book editions, improving detection accuracy.
• The end-to-end framework combining pre-trained models, active learning, and R-Drop reduced labeling cost while achieving superior F1 and recall scores across all three datasets.