[Paper Review] Unsupervised Text Generation by Learning from Search

Jingjing Li, Zichao Li+4 more|arXiv (Cornell University)|Jul 9, 2020

Topic ModelingComputer Science54 references31 citations

TL;DR

tgls combines a strong simulated annealing search with a learnable GPT2-based generator to perform unsupervised text generation, achieving state-of-the-art results on paraphrase generation and competitive performance on text formalization.

ABSTRACT

In this work, we present TGLS, a novel framework to unsupervised Text Generation by Learning from Search. We start by applying a strong search algorithm (in particular, simulated annealing) towards a heuristically defined objective that (roughly) estimates the quality of sentences. Then, a conditional generative model learns from the search results, and meanwhile smooth out the noise of search. The alternation between search and learning can be repeated for performance bootstrapping. We demonstrate the effectiveness of TGLS on two real-world natural language generation tasks, paraphrase generation and text formalization. Our model significantly outperforms unsupervised baseline methods in both tasks. Especially, it achieves comparable performance with the state-of-the-art supervised methods in paraphrase generation.

Motivation & Objective

• Address the challenge of unsupervised text generation without parallel data.
• Propose a learning-from-search framework (tgls) that alternates SA search and neural learning.
• Leverage large pretrained models (GPT-2 for fluency and RoBERTa for semantics) within tgls.
• Demonstrate effectiveness on paraphrase generation and text formalization tasks.
• Show that iterative search-learning bootstraps performance and improves inference efficiency.

Proposed method

• Use simulated annealing (SA) with a neural proposal to explore the sentence space.
• Define a product-of-experts scoring function combining fluency (s_lm) and semantics (s_semantic) and task-specific scorers (s_task).
• Fine-tune GPT-2 on SA-generated pseudo-references using word-level cross-entropy loss (CE).
• Adopt RoBERTa-based word- and sentence-level semantic scorers to inform the search and scoring.
• In a second stage, perform beam search to initialize SA and apply sequence-level max-margin (MM) learning to refine GPT-2 with positive/negative samples.
• In inference, deploy the fine-tuned GPT-2 without SA to generate text autoregressively.
• Incorporate two pretrained models: GPT-2 for likelihood/fl uency and RoBERTa for semantic evaluation and word-level prediction.

Experimental results

Research questions

• RQ1 Can unsupervised text generation reach competitive performance by learning from search results?
• RQ2 Does an iterative loop of search and learning improve generation quality beyond a single-stage approach?
• RQ3 How can large pretrained models be effectively integrated into a search-based unsupervised generation framework?
• RQ4 Is the framework robust across paraphrase generation and text formalization tasks?

Key findings

• tgls achieves iBLEU 17.48 and BLEU 25.00 on paraphrase generation, setting a new unsupervised state-of-the-art in this task.
• tgls outperforms previous unsupervised methods and approaches supervised performance in paraphrase generation (1.71 iBLEU gap to supervision).
• For text formalization, tgls outperforms existing unsupervised baselines on the GYAFC-style setting, though supervised methods still lead.
• Ablation studies show that CE learning improves SA results, and MM learning provides stronger gains when combined with SA in the second stage.
• Inference with the learned GPT-2 (without SA) is 6–10x faster than the original search-based methods, showing efficiency benefits.