[论文解读] Sequence-to-Sequence Data Augmentation for Dialogue Language Understanding

Yutai Hou, Yijia Liu+2 more|arXiv (Cornell University)|Jul 4, 2018

Topic Modeling参考文献 43被引用 117

一句话总结

本文提出基于 seq2seq 的数据增强框架，用于任务导向对话中的语言理解，使用去词汇化的 utterances 和多样性秩来生成多样且语义对齐的变体，在训练数据有限时提升 LU 性能。

ABSTRACT

In this paper, we study the problem of data augmentation for language understanding in task-oriented dialogue system. In contrast to previous work which augments an utterance without considering its relation with other utterances, we propose a sequence-to-sequence generation based data augmentation framework that leverages one utterance's same semantic alternatives in the training data. A novel diversity rank is incorporated into the utterance representation to make the model produce diverse utterances and these diversely augmented utterances help to improve the language understanding module. Experimental results on the Airline Travel Information System dataset and a newly created semantic frame annotation on Stanford Multi-turn, Multidomain Dialogue Dataset show that our framework achieves significant improvements of 6.38 and 10.04 F-scores respectively when only a training set of hundreds utterances is represented. Case studies also confirm that our method generates diverse utterances.

研究动机与目标

• Motivate data augmentation to address limited LU training data in task-oriented dialogue systems.
• Propose a fully data-driven seq2seq augmentation framework that exploits utterances sharing the same semantic frame.
• Introduce a diversity rank to encourage diverse augmented utterances and filter training pairs accordingly.
• Demonstrate effectiveness on ATIS and Stanford Multi-turn/Multidomain Dialogue Dataset with small training sets.

提出的方法

• Delexicalise input utterances by replacing slot values with semantic labels to form d and generate lexical/ syntactic variants d' via a seq2seq model.
• Train the seq2seq model conditioned on a diversity rank k appended to the input (d, k) using an attention-based encoder–decoder with input-feeding.
• Compute a diversity score between utterance pairs using EditDistance and a length difference penalty, and assign ranks to guide generation and filtering.
• Incorporate the diversity rank as an additional token to steer generation toward diverse variants.
• Filter training pairs to keep only the more diverse half of candidates when training the seq2seq model (D_seq2seq).
• Surface realizations map delexicalised slots back to actual slot values using context-aware mappings; unk tokens are replaced via attention scores.

实验结果

研究问题

• RQ1 How can we generate diverse yet semantically equivalent utterances for LU when labeled data is scarce?
• RQ2 Can a data-driven seq2seq approach leveraging same-semantic-frame utterances improve slot filling performance in LU?
• RQ3 What role does explicitly modeling diversity play in generating useful augmentation data for LU?
• RQ4 How does the proposed augmentation perform across data-scarce scenarios and different domains?

主要发现

• Significant LU gains when training data is scarce: 6.38 F-score improvement on ATIS with 129 utterances, and 2.87 F-score on ATIS with 515 utterances (medium proportion).
• On Stanford dialogue data with 100 utterances, average improvement is 10.04 across three domains; with 500 utterances, improvement is 0.47.
• The approach yields diverse, syntactically different utterances while preserving the semantic frame, as shown in case studies.
• Ablation shows seq2seq generation, diversity ranks, and filtering each contribute to performance; removing any component reduces F-score.
• Filtering less diverse pairs is important to avoid noise, even though it reduces the total number of generated utterances.