[Paper Review] Simpler and Faster Learning of Adaptive Policies for Simultaneous Translation

Baigong Zheng, Renjie Zheng+2 more|arXiv (Cornell University)|Sep 4, 2019

Natural Language Processing Techniques9 references4 citations

TL;DR

This paper proposes a supervised learning framework to train adaptive policies for simultaneous translation using oracle READ/WRITE action sequences derived from parallel text, enabling faster, more stable training and improved translation quality without retraining the underlying NMT model. The method achieves higher BLEU scores than prior approaches at low latency, with training completed in ~12 hours using a single GPU.

ABSTRACT

Simultaneous translation is widely useful but remains challenging. Previous work falls into two main categories: (a) fixed-latency policies such as Ma et al. (2019) and (b) adaptive policies such as Gu et al. (2017). The former are simple and effective, but have to aggressively predict future content due to diverging source-target word order; the latter do not anticipate, but suffer from unstable and inefficient training. To combine the merits of both approaches, we propose a simple supervised-learning framework to learn an adaptive policy from oracle READ/WRITE sequences generated from parallel text. At each step, such an oracle sequence chooses to WRITE the next target word if the available source sentence context provides enough information to do so, otherwise READ the next source word. Experiments on GermanEnglish show that our method, without retraining the underlying NMT model, can learn flexible policies with better BLEU scores and similar latencies compared to previous work.

Motivation & Objective

• To address the instability and inefficiency of reinforcement learning in training adaptive policies for simultaneous translation.
• To overcome the limitations of fixed-latency policies, which require guessing future content due to syntactic divergence in language pairs.
• To develop a method that learns flexible, controllable policies without retraining the underlying NMT model.
• To enable a single policy model to adapt to different latency requirements at inference time.
• To generate high-quality, oracle-based action sequences (READ/WRITE) from parallel sentence pairs for supervised policy training.

Proposed method

• Generate oracle action sequences (READ/WRITE) for each parallel sentence pair using a pre-trained NMT model, where actions are determined by whether the current source context suffices to predict the next target word.
• Define a READ action when insufficient context exists to predict the next target word, and a WRITE action when sufficient context is available.
• Train a supervised policy model using cross-entropy loss on the generated action sequences, directly predicting the next action based on the current source and target context.
• Control latency at inference by adjusting a threshold ρ that determines when to issue a WRITE action based on confidence in the next target word.
• Use beam search during inference with the learned policy, maintaining controllability and efficiency.
• Leverage the pre-trained NMT model for decoding, avoiding retraining while achieving superior performance.

Experimental results

Research questions

• RQ1 Can a supervised learning framework effectively train adaptive policies for simultaneous translation without reinforcement learning?
• RQ2 Can oracle-generated action sequences from parallel text lead to better policy learning than RL-based or heuristic methods?
• RQ3 Does the proposed method achieve higher translation quality (BLEU) with lower latency compared to fixed-latency wait-k policies?
• RQ4 Can a single policy model generalize across different latency requirements without retraining?
• RQ5 How does the training stability and efficiency of supervised learning compare to reinforcement learning in this setting?

Key findings

• The proposed method achieves higher BLEU scores than WIW, WID, RL-based, and test-time wait-k methods at low latency, without retraining the underlying NMT model.
• The method converges faster and more stably than reinforcement learning, with training completed in approximately 12 hours using a single GPU.
• The RL baseline required over 600 hours using 8 GPUs, highlighting the efficiency advantage of the supervised approach.
• The method maintains strong performance across different latency settings by adjusting the threshold ρ, eliminating the need for multiple models.
• The training process is stable, with no significant divergence or instability observed, unlike the RL baseline which occasionally achieved high BLEU at the cost of very high latency.
• The method outperforms test-time wait-k models, which were previously reported to suffer from short outputs or excessive punctuation for small k values.