[Paper Review] Learning Alternative Name Spellings

Jeffrey Sukharev, Leonid Zhukov+1 more|arXiv (Cornell University)|May 7, 2014

Data Quality and Management10 references3 citations

TL;DR

This paper proposes a character-level machine translation (MT) approach to learn alternative spellings of surnames, leveraging large-scale genealogical data and user search query logs to create a crowd-sourced training set. The method significantly outperforms traditional phonetic and string similarity techniques in precision and recall, with the Moses 5-gram MT model achieving the best performance on both search and records datasets.

ABSTRACT

Name matching is a key component of systems for entity resolution or record linkage. Alternative spellings of the same names are a com- mon occurrence in many applications. We use the largest collection of genealogy person records in the world together with user search query logs to build name matching models. The procedure for building a crowd-sourced training set is outlined together with the presentation of our method. We cast the problem of learning alternative spellings as a machine translation problem at the character level. We use in- formation retrieval evaluation methodology to show that this method substantially outperforms on our data a number of standard well known phonetic and string similarity methods in terms of precision and re- call. Additionally, we rigorously compare the performance of standard methods when compared with each other. Our result can lead to a significant practical impact in entity resolution applications.

Motivation & Objective

• To address the challenge of identifying credible alternative spellings of surnames in entity resolution systems.
• To develop a data-driven method that learns name variants from real user behavior in genealogy databases and search logs.
• To improve search accuracy in genealogical applications by generating high-precision, ranked lists of alternative name spellings.
• To evaluate and compare the performance of machine translation, phonetic, and string similarity methods using rigorous information retrieval metrics.
• To provide a scalable, practical solution for entity resolution that captures real-world spelling variations beyond standard algorithms.

Proposed method

• The authors frame the problem of learning alternative name spellings as a character-level machine translation task, treating name variants as translations of a source name.
• They train a neural machine translation model using the Moses toolkit, with n-gram language models (5-gram and 6-gram) to improve translation quality.
• Training data is constructed from two sources: (1) user-generated family tree links to scanned records, and (2) query reformulations from logged user sessions.
• The dataset is pre-processed to remove PII, retain only last name pairs, and filter for high-quality name matches.
• Evaluation is conducted using precision-recall curves and confidence intervals derived from 10-fold cross-validation, with elliptical density contours for statistical significance.
• Performance is compared against standard methods including Soundex, NYSIIS, Jaro-Winkler, Levenshtein, and Phonex using standard information retrieval evaluation methodology.

Experimental results

Research questions

• RQ1 Can a character-level machine translation model outperform traditional phonetic and string similarity methods in identifying alternative name spellings?
• RQ2 How do different phonetic encoding methods (e.g., NYSIIS, Soundex) compare in performance on real-world genealogical name variation data?
• RQ3 To what extent do query reformulation logs and record linkage data contribute to building a high-quality training set for alternative spelling detection?
• RQ4 What is the relative performance of various n-gram language model configurations in the MT framework for name variant generation?
• RQ5 Does the MT-based approach generalize across different data sources, such as search logs versus record databases?

Key findings

• The Moses 5-gram machine translation model significantly outperformed all other methods in both precision and recall on both the 'search' and 'records' datasets.
• The MT model achieved the highest area under the precision-recall curve, with confidence bands indicating statistically superior performance.
• NYSIIS phonetic encoding outperformed other phonetic methods, including Soundex and Phonex, on this dataset, contradicting findings from prior work.
• Levenshtein and Jaro-Winkler similarity measures performed better than other phonetic methods but were still outperformed by the MT approach.
• The 5-gram and 6-gram MT models showed slightly better performance on the 'search' dataset, while performance differences among MT models were minimal on the 'records' dataset.
• Confidence intervals for the top-performing MT model overlapped only slightly with those of Jaro-Winkler and Levenshtein, indicating a clear performance gap from other methods.