[Paper Review] Sequence-to-Sequence Learning as Beam-Search Optimization

Sam Wiseman, Alexander M. Rush|arXiv (Cornell University)|Jun 9, 2016

Natural Language Processing TechniquesComputer Science41 references117 citations

TL;DR

The paper introduces beam-search optimization (BSO) for seq2seq models, training to score sequences rather than words with LaSO-inspired loss, yielding improvements on word ordering, parsing, and machine translation compared to standard seq2seq baselines.

ABSTRACT

Sequence-to-Sequence (seq2seq) modeling has rapidly become an important general-purpose NLP tool that has proven effective for many text-generation and sequence-labeling tasks. Seq2seq builds on deep neural language modeling and inherits its remarkable accuracy in estimating local, next-word distributions. In this work, we introduce a model and beam-search training scheme, based on the work of Daume III and Marcu (2005), that extends seq2seq to learn global sequence scores. This structured approach avoids classical biases associated with local training and unifies the training loss with the test-time usage, while preserving the proven model architecture of seq2seq and its efficient training approach. We show that our system outperforms a highly-optimized attention-based seq2seq system and other baselines on three different sequence to sequence tasks: word ordering, parsing, and machine translation.

Motivation & Objective

• Address exposure bias in seq2seq training by aligning training with test-time decoding.
• Mitigate loss-evaluation mismatch by using sequence-level costs during training.
• Enable hard sequence constraints during generation via constrained beam search.
• Maintain model architecture and training efficiency while scoring entire sequences instead of individual words.

Proposed method

• Replace word-level probability training with non-probabilistic sequence scoring function f(w_t, h_{t-1}, x).
• Adopt a LaSO-like beam-search training scheme that penalizes gold sequences when they fall off the beam (BSO).
• Forward pass finds margin violations using beam search to construct candidate sequences; backward pass back-propagates through sequences that diverge from gold prefixes.
• Allow hard constraints by customizing the successor function succ to enforce valid sequences during training (ConBSO).
• Pre-train with standard word-level cross-entropy to enable effective learning of the non-local model.
• Use dropout and curriculum beam strategies to stabilize training and improve convergence.

Experimental results

Research questions

• RQ1 Can a non-probabilistic sequence-scoring seq2seq model, trained with beam-search-inspired loss, outperform standard word-level seq2seq training across diverse tasks?
• RQ2 Does LaSO-inspired training with beam search mitigate exposure bias and label bias in seq2seq models?
• RQ3 Do hard-constraint (ConBSO) and constrained beam search further improve performance on structured output tasks such as parsing?
• RQ4 How does beam size during training (K_tr) relate to decoding beam size at test time (K_te) in terms of performance?
• RQ5 What are the practical training costs and scalability implications of BS0 compared to traditional seq2seq training?

Key findings

• Word Ordering: BLEU improves from 25.2 (seq2seq) to 28.0 (BSO) and 28.6 (ConBSO) at K_te=1,5,10 respectively.
• Dependency Parsing: BSO reaches 91.00/87.18 (UAS/LAS) at K_te=5 and 91.17/87.41 at K_te=10, improving over seq2seq.
• Machine Translation: BLEU improves from 22.53 (seq2seq) to 23.83 (BSO) at K_te=1 and to 26.36/25.48 at K_te=5/10 with SB- Delta settings.
• Constrained training (ConBSO) often yields the strongest gains, e.g., in word ordering and parsing.
• Beam-size considerations: larger training beams can improve test-time performance but require careful tuning (curriculum beam).
• Training costs scale with beam size but remain manageable with shared BRNN computations; approximately a constant factor increase (~3.3x) for forward/backward passes observed on MT task.