[Paper Review] Communicative need modulates competition in language change

Andres Karjus, Richard A. Blythe+2 more|arXiv (Cornell University)|Jun 16, 2020

Language and cultural evolution72 references4 citations

TL;DR

This paper proposes a language-agnostic method to quantify competition between linguistic elements in diachronic corpora, showing that communicative need—measured via topic-level fluctuations—modulates whether near-synonymous words directly compete or coexist. The key finding is that words in topics with stable communicative need tend to compete directly, leading to replacement, while those in rising topics coexist, suggesting competition operates not only between words but also between semantic subspaces or topics.

ABSTRACT

All living languages change over time. The causes for this are many, one being the emergence and borrowing of new linguistic elements. Competition between the new elements and older ones with a similar semantic or grammatical function may lead to speakers preferring one of them, and leaving the other to go out of use. We introduce a general method for quantifying competition between linguistic elements in diachronic corpora which does not require language-specific resources other than a sufficiently large corpus. This approach is readily applicable to a wide range of languages and linguistic subsystems. Here, we apply it to lexical data in five corpora differing in language, type, genre, and time span. We find that changes in communicative need are consistently predictive of lexical competition dynamics. Near-synonymous words are more likely to directly compete if they belong to a topic of conversation whose importance to language users is constant over time, possibly leading to the extinction of one of the competing words. By contrast, in topics which are increasing in importance for language users, near-synonymous words tend not to compete directly and can coexist. This suggests that, in addition to direct competition between words, language change can be driven by competition between topics or semantic subspaces.

Motivation & Objective

• To develop a general, language-agnostic method for quantifying competition between linguistic elements in large-scale diachronic corpora without requiring language-specific resources.
• To investigate how communicative need—defined as the changing importance of topics in discourse—affects competition dynamics between near-synonymous words.
• To test the hypothesis that competition is not only between individual words but also between semantic subspaces or topics of conversation.
• To demonstrate that the proposed method yields consistent, comparable results across diverse corpora in terms of language, genre, type, and time span.
• To explore the implications of topic-level communicative need for understanding lexical innovation, replacement, and coexistence in language change.

Proposed method

• The method uses a topical advection model to track changes in the prominence of semantic topics over time, operationalizing communicative need as the rate of change in topic frequency.
• Competition between linguistic elements is quantified using a dynamic competition model that measures the degree of direct substitution between near-synonymous words based on their co-occurrence and frequency trends.
• The approach relies solely on large-scale diachronic corpora, avoiding the need for part-of-speech tagging, lemmatization, or other language-specific preprocessing.
• The model identifies semantic subspaces by clustering co-occurring words in context windows, enabling topic-level analysis of lexical dynamics.
• The method applies a smoothed frequency normalization across time bins to reduce noise and improve detection of competition patterns.
• The model's performance is validated across five diverse corpora, including the Corpus of Historical American English (COHA), showing consistent results across languages and genres.

Experimental results

Research questions

• RQ1 How does communicative need—measured as the stability or growth of a topic's importance—affect the competition between near-synonymous words?
• RQ2 To what extent do new lexical items replace older synonyms directly, and when do they coexist instead?
• RQ3 Can competition between linguistic elements be modeled quantitatively without language-specific resources?
• RQ4 Is competition better understood as occurring between individual words or between semantic subspaces or topics?
• RQ5 How do fluctuations in topic popularity correlate with lexical innovation and replacement patterns?

Key findings

• Words in topics with stable communicative need show strong direct competition, leading to the eventual replacement of one synonym by another, as seen in the historical shift from 'aeroplane' to 'airplane'.
• In contrast, words like 'famed', 'famous', and 'distinguished' coexist and increase in frequency together, indicating non-competitive coexistence in topics with rising communicative need.
• The study found that when a topic's importance is increasing, multiple near-synonyms can coexist without direct competition, suggesting reduced pressure for lexical economy.
• The model successfully detected competition dynamics across five diachronic corpora, including English, German, and Swedish, demonstrating robustness across languages and genres.
• The results support the hypothesis that competition in language change is modulated by communicative need, with topic-level dynamics playing a key role in shaping lexical evolution.
• The study reveals that lexical replacement is not always driven by direct competition between words, but can be mediated by broader shifts in discourse topics and semantic subspaces.