[Paper Review] Scholar Name Disambiguation with Search-enhanced LLM Across Language

Renyu Zhao, Yunxin Chen|arXiv (Cornell University)|Nov 26, 2024

Data Quality and ManagementDecision Sciences3 citations

TL;DR

This paper proposes a search-enhanced large language model (LLM) approach for multilingual scholar name disambiguation, leveraging search engines to retrieve cross-lingual information and enhance LLM reasoning. By integrating Chinese, English, and institutional data, the method achieves 89% recall in profile matching, significantly outperforming monolingual or single-source strategies, especially for non-English scholars.

ABSTRACT

The task of scholar name disambiguation is crucial in various real-world scenarios, including bibliometric-based candidate evaluation for awards, application material anti-fraud measures, and more. Despite significant advancements, current methods face limitations due to the complexity of heterogeneous data, often necessitating extensive human intervention. This paper proposes a novel approach by leveraging search-enhanced language models across multiple languages to improve name disambiguation. By utilizing the powerful query rewriting, intent recognition, and data indexing capabilities of search engines, our method can gather richer information for distinguishing between entities and extracting profiles, resulting in a more comprehensive data dimension. Given the strong cross-language capabilities of large language models(LLMs), optimizing enhanced retrieval methods with this technology offers substantial potential for high-efficiency information retrieval and utilization. Our experiments demonstrate that incorporating local languages significantly enhances disambiguation performance, particularly for scholars from diverse geographic regions. This multi-lingual, search-enhanced methodology offers a promising direction for more efficient and accurate active scholar name disambiguation.

Motivation & Objective

• To address the limitations of existing name disambiguation methods that rely heavily on human intervention and struggle with multilingual, heterogeneous data.
• To improve disambiguation accuracy for scholars from diverse geographic and linguistic backgrounds, particularly those publishing in non-English languages.
• To leverage search engines and multilingual LLMs to enrich scholar profiles with cross-lingual, real-time, and contextually relevant information.
• To reduce dependency on manual annotation by automating profile construction and matching using enhanced retrieval and LLM reasoning.
• To validate the effectiveness of combining local language data (e.g., Chinese) with institutional and contact information for higher disambiguation recall.

Proposed method

• The method employs search-enhanced LLMs that combine the language understanding of models like GPT4o and Llama3.2 with external search engine retrieval for real-time data access.
• It constructs multi-dimensional scholar profiles using name variants (e.g., pinyin), institutional affiliations, contact details, and research keywords across languages.
• A pipeline automatically extracts and normalizes profile data from web sources, including publications, CVs, and institutional pages.
• Cross-lingual name matching is performed by aligning pinyin, Chinese, and English variants using semantic embeddings and LLM-based normalization.
• Scholar comparison uses a scoring system: 2 points for matching institutions, 3 for repeated work/education segments, and 1–4 for keyword relevance, with a 7-point threshold for identity confirmation.
• The system integrates search queries to retrieve up to 100 relevant documents per scholar, which are then summarized and compared by the LLM to infer identity.

Figure 1: The workflow of our disambiguation method.

Experimental results

Research questions

• RQ1 How does incorporating local language data (e.g., Chinese) affect the recall and accuracy of scholar name disambiguation?
• RQ2 To what extent can search-enhanced LLMs improve disambiguation performance compared to standard LLMs using only metadata?
• RQ3 Can cross-lingual profile enrichment through search retrieval reduce the need for human-in-the-loop disambiguation?
• RQ4 What is the optimal combination of data sources (e.g., name, institution, contact info) for maximizing disambiguation recall?
• RQ5 How effective is the proposed scoring-based comparison method in distinguishing between scholars with similar profiles?

Key findings

• The method achieved 89% recall when combining pinyin, Chinese institution, and contact information, significantly outperforming other strategies.
• Using Chinese institution data alone improved recall from 18% (English-only) to 63%, highlighting the value of local language data.
• The highest recall (89%) was achieved when contact information was included, indicating its strong discriminative power.
• The proposed scoring system with a 7-point threshold achieved 100% accuracy in human evaluation, confirming its reliability.
• The integration of search-enhanced LLMs reduced reliance on manual disambiguation by enabling automated, multi-source profile construction.
• The approach demonstrated superior performance for Chinese scholars, who are often underrepresented in English-only disambiguation systems.