[Paper Review] Multilingual Word Sense Disambiguation with Unified Sense Representation

Ying Su, Hongming Zhang+2 more|arXiv (Cornell University)|Oct 14, 2022

Natural Language Processing Techniques4 citations

TL;DR

This paper proposes a multilingual word sense disambiguation (MWSD) system, mBERT-UNI, that leverages unified sense representations from BabelNet and transfers annotations from high-resource languages (e.g., English) to low-resource ones via machine translation and alignment. By jointly training on multilingual data with shared sense representations, the model improves performance on low-resource languages while maintaining strong results on English, especially for rare senses.

ABSTRACT

As a key natural language processing (NLP) task, word sense disambiguation (WSD) evaluates how well NLP models can understand the lexical semantics of words under specific contexts. Benefited from the large-scale annotation, current WSD systems have achieved impressive performances in English by combining supervised learning with lexical knowledge. However, such success is hard to be replicated in other languages, where we only have limited annotations.In this paper, based on the multilingual lexicon BabelNet describing the same set of concepts across languages, we propose building knowledge and supervised-based Multilingual Word Sense Disambiguation (MWSD) systems. We build unified sense representations for multiple languages and address the annotation scarcity problem for MWSD by transferring annotations from rich-sourced languages to poorer ones. With the unified sense representations, annotations from multiple languages can be jointly trained to benefit the MWSD tasks. Evaluations of SemEval-13 and SemEval-15 datasets demonstrate the effectiveness of our methodology.

Motivation & Objective

• Address the data scarcity problem in multilingual word sense disambiguation (MWSD) by leveraging annotations from high-resource languages.
• Overcome the challenge of inconsistent lexical knowledge and annotation scarcity across multiple languages in WSD systems.
• Enable joint training across languages by constructing unified sense representations using BabelNet synsets and lexical glosses.
• Improve performance on low-resource languages without requiring additional human-annotated data.
• Enhance disambiguation of rare senses through unified sense representations that generalize across languages.

Proposed method

• Construct unified sense representations using BabelNet synsets, which group words with the same meaning across languages.
• Inject lexical knowledge (glosses) from WordNet and Wikipedia into sense representations to enrich semantic understanding.
• Transfer annotations from high-resource languages (e.g., English) to low-resource ones using machine translation and word alignment tools.
• Train a multilingual BERT-based model (mBERT-UNI) on a joint multilingual dataset combining SemCor and automatically generated multilingual corpora.
• Use joint learning across languages to improve annotation efficiency and model generalization, especially for low-resource languages.
• Integrate data generation methods like MuLaN to further boost performance by leveraging contextualized embeddings and transfer learning.

Experimental results

Research questions

• RQ1 Can unified sense representations from BabelNet improve multilingual WSD performance across low-resource languages?
• RQ2 Does joint training on multilingual data with shared sense representations lead to better disambiguation performance than monolingual training?
• RQ3 Can transferred annotations from high-resource languages effectively improve MWSD in low-resource settings without human-annotated data?
• RQ4 How does the model perform on rare senses, which are typically harder to disambiguate due to limited training instances?
• RQ5 Does joint training negatively impact performance on high-resource languages like English?

Key findings

• The mBERT-UNI model achieves state-of-the-art performance on SemEval-13 and SemEval-15 benchmarks, with a 0.24% absolute improvement on the Italian test set when jointly training with English and Italian data.
• Joint training improves performance on four out of six low-resource languages (ES, FR, DE, IT), with comparable results on English (75.70 vs. 75.76 on the ALL test set).
• The model shows significant gains on least common senses (LCS), demonstrating that unified sense representations enhance generalization for rare senses even with minimal training data.
• The use of lexical glosses in unified representations improves performance on rare senses across all tested corpora, including both translated and MuLaN-generated data.
• The system maintains strong performance on English WSD tasks when jointly trained, indicating no negative transfer effect from low-resource languages.
• The approach enables effective cross-lingual transfer by sharing a common sense space, making it compatible with other data generation techniques like MuLaN.