[Paper Review] How to reduce the search space of Entity Resolution: with Blocking or Nearest Neighbor search?

George Papadakis, Marco Fisichella+4 more|arXiv (Cornell University)|Feb 25, 2022

Data Quality and Management4 citations

TL;DR

This paper presents the first systematic comparison of blocking and nearest neighbor (NN) filtering techniques for Entity Resolution, evaluating 5 blocking workflows and 8 NN methods across 10 real-world datasets. It finds that kNN-Join—using a cardinality threshold and syntactic representations—delivers the best balance of effectiveness and efficiency, while FAISS is the most scalable due to approximate indexing.

ABSTRACT

Entity Resolution suffers from quadratic time complexity. To increase its time efficiency, three kinds of filtering techniques are typically used for restricting its search space: (i) blocking workflows, which group together entity profiles with identical or similar signatures, (ii) string similarity join algorithms, which quickly detect entities more similar than a threshold, and (iii) nearest-neighbor methods, which convert every entity profile into a vector and quickly detect the closest entities according to the specified distance function. Numerous methods have been proposed for each type, but the literature lacks a comparative analysis of their relative performance. As we show in this work, this is a non-trivial task, due to the significant impact of configuration parameters on the performance of each filtering technique. We perform the first systematic experimental study that investigates the relative performance of the main methods per type over 10 real-world datasets. For each method, we consider a plethora of parameter configurations, optimizing it with respect to recall and precision. For each dataset, we consider both schema-agnostic and schema-based settings. The experimental results provide novel insights into the effectiveness and time efficiency of the considered techniques, demonstrating the superiority of blocking workflows and string similarity joins.

Motivation & Objective

• To address the lack of systematic comparison between blocking and nearest neighbor filtering techniques in entity resolution.
• To evaluate the relative performance of major filtering techniques across diverse real-world datasets under both schema-agnostic and schema-based settings.
• To identify optimal configurations for each technique by exhaustively tuning parameters to meet recall and precision targets.
• To assess the impact of representation type (syntactic vs. semantic), threshold type (similarity vs. cardinality), and data characteristics on filtering performance.
• To provide actionable insights for practitioners on selecting the most effective and scalable filtering method for ER workloads.

Proposed method

• The study evaluates 5 blocking workflows (e.g., Standard Blocking, Pos-Blocking Workflow) and 8 nearest neighbor methods (e.g., kNN-Join, LSH variants, FAISS, SCANN) on 10 real-world ER datasets.
• Each method is tuned across thousands of parameter configurations to meet predefined recall and precision targets, ensuring fair comparison based on optimal performance per dataset.
• Two settings are evaluated: schema-agnostic (treating entire entity profiles as long text) and schema-based (focusing on top informative attributes) to assess robustness to schema heterogeneity.
• The filtering performance is measured in terms of recall, precision, and runtime, with scalability tested on synthetic datasets up to 2 million entities.
• Cardinality-based thresholds (e.g., kNN-Join) are compared against similarity-based thresholds (e.g., ε-join, LSH) to assess trade-offs in candidate set size and efficiency.
• Syntactic representations (e.g., TF-IDF, n-grams) are contrasted with dense semantic embeddings (e.g., sentence-BERT) to evaluate their suitability for filtering.

Experimental results

Research questions

• RQ1 Which filtering technique—blocking or nearest neighbor—achieves the highest recall and precision across diverse ER datasets?
• RQ2 How do cardinality-based and similarity-based thresholds compare in terms of candidate set size and filtering efficiency?
• RQ3 Does the use of syntactic or semantic representations lead to better filtering performance in real-world ER workloads?
• RQ4 How do schema-agnostic and schema-based settings affect the effectiveness and robustness of filtering techniques?
• RQ5 Which filtering method offers the best trade-off between effectiveness, efficiency, and scalability on large-scale datasets?

Key findings

• kNN-Join, which uses a cardinality threshold and syntactic representations, achieves the best overall balance of recall, precision, and efficiency, outperforming all other methods in effectiveness.
• FAISS is the most scalable filtering technique, with a runtime increase of only ~700 times when scaling from 10,000 to 2 million entities, significantly outperforming other methods.
• Cardinality-based thresholds (e.g., kNN-Join) produce far fewer candidates than similarity-based thresholds (e.g., ε-join, LSH), scaling linearly with query size instead of quadratically with input size.
• Syntactic representations consistently outperform semantic embeddings in filtering, as semantic models introduce more false positives due to out-of-vocabulary and domain-specific terms.
• Schema-agnostic settings significantly outperform schema-based settings in recall and precision, especially when using default configurations, due to better handling of missing or misplaced values.
• The default configuration of Standard Blocking Workflow (SBW) performs well in schema-agnostic settings, but kNN-Join still surpasses it in almost all cases due to its linear candidate growth and ease of configuration.