[Paper Review] Translating Translationese: A Two-Step Approach to Unsupervised Machine Translation

Nima Pourdamghani, Nada Aldarrab+3 more|arXiv (Cornell University)|Jun 11, 2019

Natural Language Processing Techniques28 references4 citations

TL;DR

This paper proposes a two-step unsupervised neural machine translation framework that first generates a 'Translationese' gloss using a bilingual dictionary and then translates it into fluent target text using a pre-trained model. The method achieves state-of-the-art or competitive BLEU scores on high-resource languages and produces good-quality translations for 14 low-resource languages without requiring parallel training data or retraining for each new source language.

ABSTRACT

Given a rough, word-by-word gloss of a source language sentence, target language natives can uncover the latent, fully-fluent rendering of the translation. In this work we explore this intuition by breaking translation into a two step process: generating a rough gloss by means of a dictionary and then `translating' the resulting pseudo-translation, or `Translationese' into a fully fluent translation. We build our Translationese decoder once from a mish-mash of parallel data that has the target language in common and then can build dictionaries on demand using unsupervised techniques, resulting in rapidly generated unsupervised neural MT systems for many source languages. We apply this process to 14 test languages, obtaining better or comparable translation results on high-resource languages than previously published unsupervised MT studies, and obtaining good quality results for low-resource languages that have never been used in an unsupervised MT scenario.

Motivation & Objective

• To develop a fast, scalable unsupervised machine translation system that does not require retraining for new source languages.
• To address the challenge of low-resource language translation where parallel monolingual data is scarce or nonexistent.
• To improve translation quality by decoupling adequacy (glossing) from fluency (translation) using a modular pipeline.
• To enable rapid deployment of unsupervised MT systems using only off-the-shelf tools and a source-to-target dictionary.
• To demonstrate strong performance on both high-resource and previously untested low-resource languages in unsupervised settings.

Proposed method

• The method uses a pre-trained Translationese-to-target translator model trained on a diverse mix of parallel data where the source side is converted into Translationese using a bilingual dictionary.
• A source-to-target bilingual dictionary is constructed on-demand using off-the-shelf word embeddings and alignment techniques, without requiring parallel sentence-level data.
• The source text is glossed into 'Translationese'—a word-for-word pseudo-translation—using the dictionary, preserving semantic content while lacking fluency.
• The pre-trained translator model then converts the Translationese into fluent, natural-sounding target language output.
• The system leverages monolingual data for word embedding training and uses unsupervised dictionary learning via cross-lingual embedding alignment.
• The entire pipeline avoids retraining or fine-tuning when adapting to new source languages, enabling rapid deployment.

Experimental results

Research questions

• RQ1 Can a two-step pipeline combining dictionary-based glossing and pre-trained translation modeling achieve competitive unsupervised machine translation performance?
• RQ2 How well can this method generalize to low-resource languages that have not been used in prior unsupervised MT systems?
• RQ3 Can the system achieve high-quality translation without requiring parallel training data or language-specific tuning for each target language?
• RQ4 To what extent does the quality of the initial Translationese gloss affect the final translation output?
• RQ5 Can the method outperform existing unsupervised NMT approaches that rely on back-translation or pivot-based methods?

Key findings

• The method achieves a BLEU score of 21.0 on newstest2014 for French, outperforming all prior unsupervised NMT methods except Lample et al. (2018c)’s transformer model.
• For Russian, the method achieves a BLEU score of 12.0, which is significantly better than previous unsupervised methods and comparable to state-of-the-art results.
• On ten low-resource languages (e.g., Indonesian, Polish, Portuguese, Catalan), the method produces acceptable BLEU scores ranging from 13.7 to 23.1, demonstrating its viability for previously untested languages.
• The system achieves better results than Kim et al. (2018), the closest comparable method, across all tested languages, including French and German.
• The Translationese stage alone achieves a BLEU score of 11.6 for French, indicating that the glossing step preserves adequate semantic content.
• The method requires no retraining or hyperparameter tuning when adapting to new source languages, enabling rapid deployment across 14 languages with a single pre-trained model.