[Paper Review] Monotonic Simultaneous Translation with Chunk-wise Reordering and Refinement

HyoJung Han, Seokchan Ahn+4 more|arXiv (Cornell University)|Oct 18, 2021

Natural Language Processing Techniques44 references4 citations

TL;DR

This paper proposes a method to restructure standard full-sentence translation corpora into monotonic, chunk-wise reordered and refined parallel data for simultaneous machine translation (SimulMT). By leveraging word alignment and non-autoregressive translation, the approach generates target-side sequences that follow source word order more closely, enabling wait-$k$ models to produce more monotonic, lower-latency translations with improved BLEU scores across multiple language pairs, especially in challenging pairs like English-Korean.

ABSTRACT

Recent work in simultaneous machine translation is often trained with conventional full sentence translation corpora, leading to either excessive latency or necessity to anticipate as-yet-unarrived words, when dealing with a language pair whose word orders significantly differ. This is unlike human simultaneous interpreters who produce largely monotonic translations at the expense of the grammaticality of a sentence being translated. In this paper, we thus propose an algorithm to reorder and refine the target side of a full sentence translation corpus, so that the words/phrases between the source and target sentences are aligned largely monotonically, using word alignment and non-autoregressive neural machine translation. We then train a widely used wait-k simultaneous translation model on this reordered-and-refined corpus. The proposed approach improves BLEU scores and resulting translations exhibit enhanced monotonicity with source sentences.

Motivation & Objective

• To address the challenge of non-monotonic translation in SimulMT models trained on standard full-sentence corpora, which often leads to excessive anticipation or poor latency.
• To improve translation quality and monotonicity in SimulMT by transforming standard parallel corpora into monotonically aligned, chunk-wise reordered, and fluently refined training data.
• To investigate whether enhancing monotonicity in the training data leads to better monotonicity and performance in wait-$k$ SimulMT models.
• To evaluate the effectiveness of the proposed method across diverse language pairs, particularly those with significant word order differences such as English-Korean and English-Japanese.

Proposed method

• The method first segments source and target sentences into chunks using fixed-length or alignment-aware strategies to preserve semantic coherence.
• It then reorders target segments to align monotonically with source segments based on word alignment information, ensuring that target words follow the source's temporal order.
• A non-autoregressive neural machine translation (NAT) model is fine-tuned to refine the reordered target sequences, improving fluency and syntactic correctness while preserving monotonic alignment.
• The refined, monotonic parallel corpus is used to pre-train or fine-tune standard wait-$k$ SimulMT models, which generate translations incrementally with bounded delay.
• The approach is evaluated across multiple language pairs, including EnKo, EnJa, EnZh, and DeEn, using BLEU and Kendall’s $\tau$ as metrics for translation quality and monotonicity.
• The method is compared against baseline wait-$k$ models trained on standard offline corpora, with ablation studies on chunking strategies and refinement techniques.

Experimental results

Research questions

• RQ1 Can reordering and refining standard full-sentence parallel corpora improve monotonicity in SimulMT models?
• RQ2 How does the degree of word order difference between source and target languages affect the effectiveness of the proposed method?
• RQ3 To what extent does training SimulMT models on the restructured monotonic corpus improve BLEU scores and monotonicity compared to standard training?
• RQ4 Does the performance gain vary across language pairs with different average sentence lengths and word order alignments?
• RQ5 How does the choice of chunking strategy (fixed-length vs. alignment-aware) impact the final translation quality and monotonicity?

Key findings

• The proposed method improved BLEU scores by approximately 8% over the offline baseline in the EnKo language pair, with consistent gains in EnZh (+3%) and DeEn (+2%).
• Monotonicity, measured by Kendall’s $\tau$, increased significantly in the restructured training data, especially in language pairs with high word order divergence such as EnKo.
• Wait-$k$ models trained on the restructured corpus achieved higher BLEU scores on online references than on offline references, indicating better alignment with human-like monotonic interpretation.
• The improvement in monotonicity was proportional to the monotonicity gain in the training data, confirming that training data monotonicity directly influences model behavior.
• The method showed limited improvement in EnJa, likely due to short average sentence length and high emission rate, which reduces the benefit of chunking and reordering.
• The Fixed + NAT and AlignAw + NAT refinement strategies both enhanced fluency, but AlignAw + NAT yielded better monotonicity, especially at higher $k$ values.