[Paper Review] Code-Switching for Enhancing NMT with Pre-Specified Translation

Kai Song, Yue Zhang+4 more|arXiv (Cornell University)|Apr 19, 2019

Natural Language Processing Techniques29 references43 citations

TL;DR

This paper proposes a data-augmentation method that creates code-switched training data by replacing source phrases with their pre-specified translations, enabling the NMT model to copy target-language phrases directly and improving constrained translation without harming unconstrained words. It demonstrates consistent BLEU gains over placeholder and lexical-constraint methods on En-Ru and Ch-En across multiple domains, and preserves performance on non-constrained sentences.

ABSTRACT

Leveraging user-provided translation to constrain NMT has practical significance. Existing methods can be classified into two main categories, namely the use of placeholder tags for lexicon words and the use of hard constraints during decoding. Both methods can hurt translation fidelity for various reasons. We investigate a data augmentation method, making code-switched training data by replacing source phrases with their target translations. Our method does not change the MNT model or decoding algorithm, allowing the model to learn lexicon translations by copying source-side target words. Extensive experiments show that our method achieves consistent improvements over existing approaches, improving translation of constrained words without hurting unconstrained words.

Motivation & Objective

• Motivate constraints in domain-specific NMT and the limitations of placeholder and lexical-constraint methods.
• Introduce a code-switching data augmentation approach to teach NMT to copy pre-specified translations.
• Enhance copying with shared embeddings and a pointer network integrated into the decoder.
• Demonstrate improvements over baselines on En-Ru and Ch-En across multiple domains.

Proposed method

• Construct augmented parallel data by replacing source phrases with their target translations using an SMT phrase table.
• Preprocess decoding by replacing source phrases with pre-specified translations without modifying the decoder.
• Share target embeddings with source-side target words to align embeddings for copying.
• Integrate a pointer network into the decoder to copy source-side target words.
• Mix augmented data with original data to train a standard Transformer without changing decoding algorithms.
• Evaluate with BLEU on En-Ru and Ch-En, comparing against placeholder and lexical-constraint baselines.

Experimental results

Research questions

• RQ1 Can code-switched training data enable NMT to copy pre-specified translations effectively during decoding?
• RQ2 Do shared embeddings and a pointer network improve copying and overall translation quality?
• RQ3 How does this method compare to placeholder and lexical-constrained decoding across domains?
• RQ4 Does the approach preserve translation quality for non-constrained sentences?

Key findings

• The method yields consistent improvements over placeholder and lexical-constraint baselines on En-Ru and Ch-En across multiple test sets and domains.
• In the English–Russian news domain, average BLEU gains are 3.48 over placeholder and 2.94 over lexical constraints.
• In the English–Russian e-commerce domain, average BLEU gains are 1.34 over placeholder and 2.63 over lexical constraints.
• In the Chinese–English spoken-domain set, average BLEU gains are 1.35 over placeholder and 0.42 over lexical constraints.
• Shared embeddings and the pointer network improve copying success and translation quality.
• The approach preserves competitiveness on non-code-switched input, with comparable or slightly improved BLEU on some non-replaced sentences.