[Paper Review] Is this word borrowed? An automatic approach to quantify the likeliness of borrowing in social media

Jasabanta Patro, Bidisha Samanta+4 more|arXiv (Cornell University)|Mar 15, 2017

Linguistics, Language Diversity, and Identity23 references3 citations

TL;DR

This paper proposes a novel computational framework to automatically quantify the likelihood of word borrowing in social media, using context-based clustering and three new metrics—UUR, UUR-adj, and UUR-adj-2—derived from user-level word usage patterns in English-Hindi code-mixed tweets. The method achieves a Spearman’s rank correlation of 0.62 with human-annotated ground truth, more than double the baseline (0.26), and shows stronger performance for younger users and low code-mixing users, indicating early detection of borrowing trends.

ABSTRACT

Code-mixing or code-switching are the effortless phenomena of natural switching between two or more languages in a single conversation. Use of a foreign word in a language; however, does not necessarily mean that the speaker is code-switching because often languages borrow lexical items from other languages. If a word is borrowed, it becomes a part of the lexicon of a language; whereas, during code-switching, the speaker is aware that the conversation involves foreign words or phrases. Identifying whether a foreign word used by a bilingual speaker is due to borrowing or code-switching is a fundamental importance to theories of multilingualism, and an essential prerequisite towards the development of language and speech technologies for multilingual communities. In this paper, we present a series of novel computational methods to identify the borrowed likeliness of a word, based on the social media signals. We first propose context based clustering method to sample a set of candidate words from the social media data.Next, we propose three novel and similar metrics based on the usage of these words by the users in different tweets; these metrics were used to score and rank the candidate words indicating their borrowed likeliness. We compare these rankings with a ground truth ranking constructed through a human judgment experiment. The Spearman's rank correlation between the two rankings (nearly 0.62 for all the three metric variants) is more than double the value (0.26) of the most competitive existing baseline reported in the literature. Some other striking observations are, (i) the correlation is higher for the ground truth data elicited from the younger participants (age less than 30) than that from the older participants, and (ii )those participants who use mixed-language for tweeting the least, provide the best signals of borrowing.

Motivation & Objective

• To develop an automatic method for quantifying the likelihood that a foreign word in a language is being borrowed rather than used via code-switching.
• To identify early linguistic signals of borrowing from informal, large-scale social media data, particularly from multilingual users.
• To improve upon existing baselines by leveraging usage patterns in social media, especially from users with minimal code-mixing behavior.
• To validate the method through human-annotated ground truth data collected from diverse age groups.

Proposed method

• A context-based clustering approach is used to sample 57 candidate English words from English-Hindi code-mixed tweets, ensuring linguistic relevance and diversity.
• Three novel metrics—UUR, UUR-adj, and UUR-adj-2—are proposed, based on user-level word frequency and distribution across tweets to estimate borrowing likelihood.
• The metrics compute the uniformity of usage across users, adjusting for word frequency and user activity to reduce noise and bias.
• A human evaluation study was conducted with 58 judges to create a ground-truth ranking of borrowing likelihood for the 57 target words, stratified by age group.
• Spearman’s rank correlation is used to compare the model’s rankings against the human-annotated ground truth, with additional analysis by user code-mixing behavior.
• The method is evaluated across different user groups (high, mid, low code-mixing) to assess signal strength and robustness.

Experimental results

Research questions

• RQ1 Can social media data provide reliable early signals of word borrowing, distinct from code-switching?
• RQ2 How do usage patterns of foreign words in social media correlate with human judgments of borrowing likelihood?
• RQ3 Do users who rarely code-mix provide stronger signals for identifying borrowing compared to frequent code-switchers?
• RQ4 Is the proposed method more effective for younger users, reflecting early-stage borrowing trends?
• RQ5 How do the proposed metrics compare quantitatively to existing baselines in predicting borrowing likelihood?

Key findings

• The proposed UUR metric achieves a Spearman’s rank correlation of 0.62 with human-annotated ground truth, more than double the baseline correlation of 0.26.
• The correlation is significantly higher for younger participants (age < 30) at 0.62, suggesting the method captures early-stage borrowing trends effectively.
• The UUR metric performs best when derived from users with low code-mixing behavior, achieving a correlation of 0.65, indicating such users provide the purest signals of borrowing.
• Precision and recall metrics show consistent improvement for UUR over the baseline, especially in the SB (seldom-borrowed) and LM (likely to be borrowed) buckets.
• Macro and micro precision and recall values for UUR are consistently higher than the baseline across both young and elderly user groups.
• The method demonstrates robustness and generalizability, with strong performance across multiple evaluation schemes and ground-truth variants.