[Paper Review] A Generalized Framework of Sequence Generation with Application to Undirected Sequence Models

Elman Mansimov, Alex Wang+2 more|arXiv (Cornell University)|May 29, 2019

Natural Language Processing Techniques29 references46 citations

TL;DR

The paper proposes a generalized framework to unify decoding for directed and undirected neural sequence models, enabling constant-time and adaptive-order generation from models like BERT, and demonstrates competitive machine translation results on WMT’14 En-De with various decoding strategies.

ABSTRACT

Undirected neural sequence models such as BERT (Devlin et al., 2019) have received renewed interest due to their success on discriminative natural language understanding tasks such as question-answering and natural language inference. The problem of generating sequences directly from these models has received relatively little attention, in part because generating from undirected models departs significantly from conventional monotonic generation in directed sequence models. We investigate this problem by proposing a generalized model of sequence generation that unifies decoding in directed and undirected models. The proposed framework models the process of generation rather than the resulting sequence, and under this framework, we derive various neural sequence models as special cases, such as autoregressive, semi-autoregressive, and refinement-based non-autoregressive models. This unification enables us to adapt decoding algorithms originally developed for directed sequence models to undirected sequence models. We demonstrate this by evaluating various handcrafted and learned decoding strategies on a BERT-like machine translation model (Lample & Conneau, 2019). The proposed approach achieves constant-time translation results on par with linear-time translation results from the same undirected sequence model, while both are competitive with the state-of-the-art on WMT'14 English-German translation.

Motivation & Objective

• Motivate decoding from undirected neural sequence models (e.g., BERT) and formalize a generation-centric framework.
• Unify decoding for directed and undirected models by separating length, coordinate (position) selection, and symbol replacement.
• Derive monotonic autoregressive, non-autoregressive, and semi-autoregressive decoding as special cases within the framework.
• Develop and evaluate deterministic and learned decoding strategies for undirected models on MT tasks.
• Demonstrate constant-time decoding variants with competitive performance to autoregressive baselines.

Proposed method

• Define a generation process G with intermediate sequences Y^t and coordinate sequences Z^t and model p(G|X) via length prediction p(L|X), coordinate selection p(z^{t+1}_i|Y^{≤t},Z^t,X), and symbol replacement p(y^{t+1}_i|Y^{≤t},Z^{t+1},X).
• Show that monotonic autoregressive, non-autoregressive iterative refinement, and semi-autoregressive decoding are special cases of the framework.
• Instantiate a log-linear or learned coordinate selector to adapt generation order, including an adaptive Gibbs sampling scheme with features such as negative entropy, log-probability, and positional bias.
• Cast Gibbs sampling within the generalized framework to enable deterministic optimistic decoding via length-conditioned beam search.
• Train a coordinate-selection policy with reinforcement learning (PPO) to maximize reward based on generation quality (edit-distance changes).
• Evaluate decoding strategies on WMT’14 English-German translation using a masked translation model (pretrained BERT-like) and compare to a left-to-right autoregressive baseline.

Experimental results

Research questions

• RQ1 Can decoding from undirected sequence models be unified with directed models under a single generation-centric framework?
• RQ2 How do different coordinate selection and symbol replacement strategies affect translation quality in undirected models?
• RQ3 What are the trade-offs between deterministic and learned decoding for undirected models, in terms of BLEU and efficiency?
• RQ4 Can constant-time decoding from undirected models achieve competitive performance to linear-time or autoregressive methods?
• RQ5 Does adaptive ordering (left-to-right, outside-in, learned policies) emerge as beneficial strategies for MT with masked language models?

Key findings

• The framework unifies multiple decoding paradigms (autoregressive, non-autoregressive, semi-autoregressive) under a single generation process.
• Deterministic coordinate selection strategies (left-to-right, least-to-most, easy-first, learned) outperform uniform coordinate selection by up to about 3 BLEU on En–De and De–En.
• Beam search improves performance by about 1 BLEU over greedy decoding.
• Rescoring with an autoregressive model adds about 1 BLEU.
• Learned and left-to-right strategies provide the strongest performance, with left-to-right achieving 25.66 BLEU (En→De) and learned achieving 30.58 BLEU (De→En) on MT tasks.
• Constant-time decoding with budget T and o_t settings can match linear-time decoding and approach state-of-the-art autoregressive performance (e.g., 30.13 vs. 30.92 BLEU on De→En with certain settings).