[Paper Review] Faster Re-translation Using Non-Autoregressive Model For Simultaneous Neural Machine Translation

HyoJung Han, Sathish Reddy Indurthi+6 more|arXiv (Cornell University)|Dec 29, 2020

Natural Language Processing Techniques24 references4 citations

TL;DR

This paper proposes FReTNA, a non-autoregressive re-translation model for simultaneous neural machine translation that generates target sequences in parallel, reducing inference time by a factor of 10 compared to autoregressive re-translation (ReTA) while maintaining high translation quality. The method uses a Levenshtein Transformer to enable efficient insertions and deletions, and introduces a new stability metric, NCNE, to better capture user-perceived flicker in dynamic translations.

ABSTRACT

Recently, simultaneous translation has gathered a lot of attention since it enables compelling applications such as subtitle translation for a live event or real-time video-call translation. Some of these translation applications allow editing of partial translation giving rise to re-translation approaches. The current re-translation approaches are based on autoregressive sequence generation models (ReTA), which generate tar-get tokens in the (partial) translation sequentially. The multiple re-translations with sequential generation inReTAmodelslead to an increased inference time gap between the incoming source input and the corresponding target output as the source input grows. Besides, due to the large number of inference operations involved, the ReTA models are not favorable for resource-constrained devices. In this work, we propose a faster re-translation system based on a non-autoregressive sequence generation model (FReTNA) to overcome the aforementioned limitations. We evaluate the proposed model on multiple translation tasks and our model reduces the inference times by several orders and achieves a competitive BLEUscore compared to the ReTA and streaming (Wait-k) models.The proposed model reduces the average computation time by a factor of 20 when compared to the ReTA model by incurring a small drop in the translation quality. It also outperforms the streaming-based Wait-k model both in terms of computation time (1.5 times lower) and translation quality.

Motivation & Objective

• To address the high inference time of autoregressive re-translation (ReTA) models in simultaneous neural machine translation, which increases with input length.
• To reduce wall-clock latency in re-translation systems by replacing sequential autoregressive decoding with parallel non-autoregressive generation.
• To improve user experience by introducing a more sensitive stability metric that captures textual flickers from insertions, deletions, and replacements.
• To demonstrate that non-autoregressive re-translation can outperform both ReTA and streaming Wait-k models in both speed and translation quality.

Proposed method

• Proposes FReTNA, a non-autoregressive re-translation model based on the Levenshtein Transformer, which allows simultaneous insertion and deletion of multiple tokens during inference.
• Adapts the Levenshtein Transformer’s training objective to support correction-based generation, enabling efficient re-translation without full autoregressive re-generation.
• Introduces positional bias during inference to guide token prediction and reduce flickers, especially in low-latency scenarios.
• Employs a new stability metric, Normalized Click-N-Edit (NCNE), which quantifies textual instability by counting insertions, deletions, and replacements between consecutive translations.
• Uses a re-translation strategy where the model re-generates the entire partial translation at each new source input, but in parallel, avoiding sequential autoregressive generation.
• Trains the model using a prefix-augmented loss to improve alignment and performance on partial translations.

Experimental results

Research questions

• RQ1 Can non-autoregressive generation significantly reduce inference time in re-translation-based simultaneous neural machine translation?
• RQ2 How does the proposed FReTNA model compare to ReTA and Wait-k models in terms of translation quality and computation efficiency?
• RQ3 Does the new NCNE metric better capture user-perceived textual flickers than the existing NE metric?
• RQ4 To what extent does positional bias improve stability and reduce flickers in low-latency re-translation scenarios?

Key findings

• The proposed FReTNA model reduces average computation time by a factor of 10 compared to ReTA models, achieving significant speedup with only a small drop in translation quality.
• FReTNA outperforms the streaming Wait-k model by reducing computation time by 1.5 times while also achieving better translation quality.
• The FReTNA_pos variant with positional bias achieves lower NCNE scores than FReTNA_non_pos, indicating reduced flickers, especially in low-latency regimes.
• The NCNE metric detects more flickers than the NE metric, as it accounts for all token-level changes (insertions, deletions, replacements), not just the first point of difference.
• In low-latency settings, FReTNA_non_pos shows higher flicker due to over-prediction and subsequent deletion of tokens, which is mitigated by positional bias in FReTNA_pos.
• The model maintains high translation quality while drastically reducing inference latency, making it suitable for real-time applications on resource-constrained devices.