[Paper Review] A Hybrid Morpheme-Word Representation for Machine Translation of Morphologically Rich Languages

Minh-Thang Luong, Preslav Nakov+1 more|arXiv (Cornell University)|Nov 19, 2019

Natural Language Processing Techniques27 references43 citations

TL;DR

The paper presents a language-independent hybrid morpheme-word SMT model that uses morpheme-level translation with word-boundary awareness, combined with twin language models and phrase-table merging to improve English-Finnish translation.

ABSTRACT

We propose a language-independent approach for improving statistical machine translation for morphologically rich languages using a hybrid morpheme-word representation where the basic unit of translation is the morpheme, but word boundaries are respected at all stages of the translation process. Our model extends the classic phrase-based model by means of (1) word boundary-aware morpheme-level phrase extraction, (2) minimum error-rate training for a morpheme-level translation model using word-level BLEU, and (3) joint scoring with morpheme- and word-level language models. Further improvements are achieved by combining our model with the classic one. The evaluation on English to Finnish using Europarl (714K sentence pairs; 15.5M English words) shows statistically significant improvements over the classic model based on BLEU and human judgments.

Motivation & Objective

• Motivate improving SMT for morphologically rich languages without language-specific tools.
• Introduce a morpheme-level translation unit while respecting word boundaries throughout decoding.
• Enhance decoding with twin morpheme-token and word-token language models and MERT optimization at the word level.
• Develop and evaluate methods to merge morpheme-based and word-based translation tables.
• Demonstrate improvements on English→Finnish with large-scale Europarl data using BLEU and human judgments.

Proposed method

• Adopt a Morfessor-based unsupervised morpheme analyzer to produce PRE, STM, SUF tokens.
• Implement word boundary-aware morpheme-level phrase extraction that can extend beyond seven morpheme-tokens as long as they span ≤ seven words.
• Perform MERT at the word-token level by converting morpheme-token outputs to word sequences for BLEU calculation during training.
• Introduce twin language models (morpheme-token LM and word-token LM) and score with both during decoding.
• Merge and normalize two phrase tables from morpheme-token and word-token inputs via interpolation-style methods and a novel merging approach that preserves probabilistic integrity.
• Evaluate a twin translation model that merges PT_m and PT_w→m into PT_m+phr and compare with baselines.

Experimental results

Research questions

• RQ1 Can morpheme-level translation with word boundary awareness outperform standard word-level SMT for morphologically rich languages?
• RQ2 Does incorporating a morpheme-token LM alongside a word-token LM improve decoding for morphologically rich target languages?
• RQ3 Does merging phrase tables from morpheme-token and word-token translations provide additional gains over single-system SMT?
• RQ4 How do morphological enhancements interact with training data size in improving translation quality for English→Finnish?

Key findings

• Morpheme-based enhancements yield BLEU gains over the morpheme baseline, with tune (MERT at word level) and a combination of phr and lm delivering the largest improvements.
• The full system with morpheme boundaries, twin LMs, and phrase-table merging achieves BLEU 14.82 and m-BLEU 55.64, outperforming both baseline systems (BLEU 14.58 and m-BLEU 53.05 for word baseline; BLEU 14.08 and m-BLEU 55.26 for morpheme baseline).
• Interpolation-based and ourMethod merging approaches significantly improve over the morpheme-only baseline, with ourMethod achieving the highest full-dataset BLEU gains (0.74 points over the m-system).
• Human judgments favor the proposed system over both m-system and w-system, with statistically significant preferences in pairwise comparisons.
• Analysis shows translations are often close to reference wordforms, suggesting improvements are meaningful beyond BLEU scores.