[Paper Review] A Survey on Contextualised Semantic Shift Detection

Stefano Montanelli, Francesco Periti|arXiv (Cornell University)|Apr 4, 2023

Language and cultural evolutionSocial Sciences104 references16 citations

TL;DR

This paper surveys contextualised semantic shift detection (CSSDetection) approaches, proposes a three-dimension classification framework (meaning representation, time-awareness, learning modality), and analyzes assessment measures, datasets, and open challenges.

ABSTRACT

Semantic Shift Detection (SSD) is the task of identifying, interpreting, and assessing the possible change over time in the meanings of a target word. Traditionally, SSD has been addressed by linguists and social scientists through manual and time-consuming activities. In the recent years, computational approaches based on Natural Language Processing and word embeddings gained increasing attention to automate SSD as much as possible. In particular, over the past three years, significant advancements have been made almost exclusively based on word contextualised embedding models, which can handle the multiple usages/meanings of the words and better capture the related semantic shifts. In this paper, we survey the approaches based on contextualised embeddings for SSD (i.e., CSSDetection) and we propose a classification framework characterised by meaning representation, time-awareness, and learning modality dimensions. The framework is exploited i) to review the measures for shift assessment, ii) to compare the approaches on performance, and iii) to discuss the current issues in terms of scalability, interpretability, and robustness. Open challenges and future research directions about CSSDetection are finally outlined.

Motivation & Objective

• Define CSSDetection and its importance for automating semantic-shift analysis over time.
• Propose a three-dimensional classification framework for CSSDetection approaches (meaning representation, time-awareness, learning modality).
• Review state-of-the-art CSSDetection methods and how they are evaluated.
• Compare approaches using shared tasks and corpora when available.
• Identify scalability, interpretability, and robustness challenges and outline future research directions.

Proposed method

• Introduce a formal workflow for CSSDetection: embedding, optional aggregation, and shift assessment.
• Classify approaches along three dimensions: meaning representation (form-based vs. sense-based), time-awareness (time-oblivious vs. time-aware), and learning modality (supervised vs. unsupervised).
• Describe and formalize semantic shift measures (e.g., cosine distance between prototypes, inverted similarity over prototypes, time-diff, average pairwise distance).
• Discuss aggregation techniques (clustering vs. averaging) and their impact on shift measurement.
• Provide a catalog of form-based and sense-based CSSDetection methods with model types, training regimes, and shift functions.
• Summarize results from shared tasks (e.g., SemEval-20 Task 1, DIACRIta-20, RuShiftEval-21, LSCDiscovery-22) and compare reported performance where available.

Experimental results

Research questions

• RQ1 How can CSSDetection approaches be systematically classified and compared?
• RQ2 What meaning representations and time-aware strategies are used in CSSDetection, and how do they impact detection and interpretability?
• RQ3 What learning paradigms (supervised vs. unsupervised) are employed in CSSDetection, and what external knowledge is utilized or avoided?
• RQ4 What semantic shift measures are used to quantify changes, and how do they perform across tasks and languages?
• RQ5 What are the current scalability, interpretability, and robustness limitations, and what future directions are suggested?

Key findings

• Most form-based CSSDetection methods are time-oblivious and rely on unsupervised learning, with averaging as the common aggregation strategy.
• Sense-based approaches use clustering to capture multiple word usages and meanings, enabling interpretation of shift across meanings.
• Cosine distance between prototypes (CD) is a widely used shift function, with alternatives like inverted similarity (PRT) and time-aware variants (TD, APD) discussed.
• Time-aware approaches typically fine-tune or adapt pre-trained models with time markers or temporal references to capture temporal dynamics.
• Shared-task evaluations (e.g., SemEval-20, DIACRIta-20, RuShiftEval-21, LSCDiscovery-22) are used to compare CSSDetection methods, though results are limited by task specifics and language.
• The survey highlights open challenges in scalability, interpretability, and robustness, and outlines future research directions in CSSDetection.