[Paper Review] Two Birds, One Stone: A Simple, Unified Model for Text Generation from Structured and Unstructured Data

Hamidreza Shahidi, Ming Li+1 more|arXiv (Cornell University)|Sep 23, 2019

Topic Modeling39 references4 citations

TL;DR

This paper proposes a simple, unified sequence-to-sequence model with attention and a copy mechanism, trained using exponential moving average (EMA), to generate text from both structured (tables) and unstructured (passages) data. Despite its simplicity, the model achieves state-of-the-art performance on both neural table-to-text generation and neural question generation tasks, demonstrating that architectural complexity is not necessary for high performance when models are carefully tuned.

ABSTRACT

A number of researchers have recently questioned the necessity of increasingly complex neural network (NN) architectures. In particular, several recent papers have shown that simpler, properly tuned models are at least competitive across several NLP tasks. In this work, we show that this is also the case for text generation from structured and unstructured data. We consider neural table-to-text generation and neural question generation (NQG) tasks for text generation from structured and unstructured data, respectively. Table-to-text generation aims to generate a description based on a given table, and NQG is the task of generating a question from a given passage where the generated question can be answered by a certain sub-span of the passage using NN models. Experimental results demonstrate that a basic attention-based seq2seq model trained with the exponential moving average technique achieves the state of the art in both tasks. Code is available at https://github.com/h-shahidi/2birds-gen.

Motivation & Objective

• To investigate whether increasingly complex neural architectures are truly necessary for text generation from structured and unstructured data.
• To evaluate whether a simple, well-tuned model can outperform or match complex models on text generation tasks.
• To unify text generation from tables (structured) and passages (unstructured) under a single model architecture.
• To demonstrate that performance gains from complex models may stem from hyperparameter tuning rather than architectural innovation.
• To challenge the prevailing trend of architectural complexity in NLP by showing that simple models with EMA can achieve state-of-the-art results.

Proposed method

• A bidirectional, attention-based sequence-to-sequence model is used as the core architecture for both tasks.
• The model incorporates a copy mechanism to handle OOV (out-of-vocabulary) words, especially useful for entity names and rare terms.
• Exponential moving average (EMA) is applied during training to stabilize and improve model weights, enhancing generalization.
• Field and position embeddings are concatenated with word embeddings to represent structured table data.
• Beam search with a beam size of 20 and length penalty of 1.75 is used during decoding for both tasks.
• The model is trained for up to 20 epochs with dropout (0.1 and 0.3 for different splits) and uses a shared vocabulary from Song et al. (2018).

Experimental results

Research questions

• RQ1 Can a simple, unified neural sequence-to-sequence model achieve state-of-the-art performance on both table-to-text generation and question generation tasks?
• RQ2 Does the use of exponential moving average (EMA) significantly improve performance on these tasks without architectural complexity?
• RQ3 Is the performance gain from complex models in NLP primarily due to architectural design or better hyperparameter tuning of simpler models?
• RQ4 Can a model without explicit linguistic features or task-specific modules still achieve SOTA results on text generation from structured and unstructured data?
• RQ5 To what extent do architectural innovations in recent models provide marginal gains compared to careful tuning of basic models?

Key findings

• The proposed simple seq2seq model with EMA achieves state-of-the-art results on the SQuAD dataset for neural question generation, with a ROUGE-L score of 45.18 ± 0.22 on split-2.
• On the WikiBio dataset for table-to-text generation, the model achieves a BLEU-4 score of 46.76 ± 0.03 and a ROUGE-4 score of 43.54 ± 0.07 when using EMA.
• The model outperforms prior complex models such as Liu et al. (2019b) and Kim et al. (2019), even though it uses a simpler architecture without task-specific modules.
• The EMA technique significantly improves performance, with a 1.36-point increase in ROUGE-L on NQG and a 0.69-point increase in BLEU-4 on table-to-text generation.
• The model achieves competitive results on BLEU-4 (16.14 ± 0.25) and METEOR (20.44 ± 0.20) on SQuAD split-2, outperforming most prior work.
• The results suggest that architectural complexity is not a prerequisite for SOTA performance, and that careful tuning of simple models can yield superior outcomes.