[Paper Review] Transliteration in Any Language with Surrogate Languages

Stephen Mayhew, Christos Christodoulopoulos+1 more|arXiv (Cornell University)|Sep 14, 2016

Natural Language Processing Techniques15 references3 citations

TL;DR

This paper proposes a method to generate transliterations in any language using Wikipedia as a surrogate training source, even for low-resource languages. By ranking Wikipedia languages based on script similarity, phonetic inventory, and learned embeddings, the approach achieves performance comparable to or better than an oracle that knows the best language in advance, with weighted voting across top candidates outperforming the oracle in some cases.

ABSTRACT

We introduce a method for transliteration generation that can produce transliterations in every language. Where previous results are only as multilingual as Wikipedia, we show how to use training data from Wikipedia as surrogate training for any language. Thus, the problem becomes one of ranking Wikipedia languages in order of suitability with respect to a target language. We introduce several task-specific methods for ranking languages, and show that our approach is comparable to the oracle ceiling, and even outperforms it in some cases.

Motivation & Objective

• To enable transliteration generation in any language, including low-resource ones, without requiring direct training data in the target language.
• To address the limitation of existing methods that rely on Wikipedia data only for languages already present in the corpus.
• To develop a task-driven language similarity metric that ranks Wikipedia languages for their suitability as surrogate training sources.
• To demonstrate that surrogate language selection can match or exceed performance of an oracle that knows the optimal language in advance.

Proposed method

• The method uses 131 Wikipedia languages with at least 200 name pairs as potential surrogate languages, denoted as set $\mathcal{W}$.
• All text is romanized using uroman to standardize scripts into Latin characters, enabling cross-lingual training and testing.
• Language similarity is computed via three metrics: script distribution similarity (cosine of character histograms), phonetic inventory similarity (F1 of IPA phonemes), and learned embeddings from a transliteration model.
• The top surrogate language is selected based on the highest similarity score, and a transliteration model is trained on English–surrogate pairs to generate transliterations into the target language $T$.
• For improved performance, multiple top surrogate languages are combined using weighted voting on n-best lists from each model.
• The approach is evaluated using Mean Reciprocal Rank (MRR) on 9 low-resource target languages, excluding the target language itself from candidate rankings to simulate low-resource conditions.

Experimental results

Research questions

• RQ1 Can a surrogate language from Wikipedia be used to train a transliteration model that performs as well as one trained on the target language itself?
• RQ2 How effective are different language similarity metrics—script, phonetic, and learned embedding—for selecting the best surrogate language?
• RQ3 Can combining predictions from multiple top surrogate languages improve performance beyond the best single surrogate or even the oracle?
• RQ4 Does script similarity, a simple and knowledge-light metric, outperform more complex phonetic or learned similarity measures in surrogate selection?
• RQ5 In low-resource settings where no training data exists for the target language, can surrogate-based methods surpass oracle performance?

Key findings

• Script similarity outperforms both phonetic and learned similarity metrics, achieving the highest average MRR across all target languages.
• The top predicted surrogate language matches the oracle's choice in 4 out of 9 target languages, and its performance is consistently close to the oracle's.
• For Newar and Uzbek, the top 5 combined surrogate languages outperform the oracle, with MRR scores of 24.81* and 44.37*, respectively, indicating the method can surpass the oracle in some cases.
• The method achieves MRR scores comparable to or better than training on the target language itself, even when the target language has no Wikipedia data.
• The use of weighted voting across the top 5 surrogate languages improves performance in 7 out of 9 languages, with two cases exceeding the oracle's performance.