[论文解读] A Survey of Natural Language Generation Techniques with a Focus on Dialogue Systems - Past, Present and Future Directions

Sashank Santhanam, Samira Shaikh|arXiv (Cornell University)|Jun 2, 2019

Topic Modeling参考文献 129被引用 34

一句话总结

本文综述从传统到深度学习的自然语言生成方法，聚焦开放域对话系统，并概述三个关键未来方向：更大上下文、个性化/人设，以及避免乏味回应，采用认知架构视角。

ABSTRACT

One of the hardest problems in the area of Natural Language Processing and Artificial Intelligence is automatically generating language that is coherent and understandable to humans. Teaching machines how to converse as humans do falls under the broad umbrella of Natural Language Generation. Recent years have seen unprecedented growth in the number of research articles published on this subject in conferences and journals both by academic and industry researchers. There have also been several workshops organized alongside top-tier NLP conferences dedicated specifically to this problem. All this activity makes it hard to clearly define the state of the field and reason about its future directions. In this work, we provide an overview of this important and thriving area, covering traditional approaches, statistical approaches and also approaches that use deep neural networks. We provide a comprehensive review towards building open domain dialogue systems, an important application of natural language generation. We find that, predominantly, the approaches for building dialogue systems use seq2seq or language models architecture. Notably, we identify three important areas of further research towards building more effective dialogue systems: 1) incorporating larger context, including conversation context and world knowledge; 2) adding personae or personality in the NLG system; and 3) overcoming dull and generic responses that affect the quality of system-produced responses. We provide pointers on how to tackle these open problems through the use of cognitive architectures that mimic human language understanding and generation capabilities.

研究动机与目标

• 提供从传统方法到深度学习方法的自然语言生成（NLG）概述。
• 将开放域对话系统作为NLG的一个关键应用进行考察。
• 识别研究差距并提出未来方向，以实现更连贯、具上下文感知和个性化的对话生成。
• 讨论认知架构如何解决对话NLG当前的局限性。

提出的方法

• 评述传统NLG体系结构及其子组件（内容确定、文档结构化、词汇化、指称生成、句子聚合、语言实现）。
• 讨论在神经模型之前用于内容选择与实现的统计方法和基于规则的方法。
• 调查深度学习方法（语言模型、编码器-解码器/Seq2Seq、注意力机制、记忆网络、基于Transformer的方法）及其对对话系统的影响。
• 突出对话生成中持续存在的挑战，如缺乏上下文编码、通用回应以及缺失人设。
• 提供未来方向的综合与潜在的受认知架构启发的解决方案。

实验结果

研究问题

• RQ1 对话系统的主要历史与当代自然语言生成方法有哪些？
• RQ2 哪些关键研究差距阻碍开放域对话系统实现连贯、上下文丰富和个性化的互动？
• RQ3 未来的NLG系统如何融入更大上下文和世界知识，以及如何整合人设或性格以提升对话质量？
• RQ4 认知架构在解决对话NLG当前局限性方面能发挥何种作用？

主要发现

• Seq2seq模型和语言模型主导了当前的对话系统方法。
• 识别出三大未来方向：融入更大上下文和世界知识、整合人设或个性，以及克服乏味、通用的回应。
• 尚存的问题包括编码更广的对话上下文、维持连贯的人设，以及产生更具吸引力且与情境相关的回应。
• 综述建议探索模仿人类语言理解与生成的认知架构，以解决基础性挑战。
• 传统的NLG组件（内容确定、结构化、词汇化、REG、聚合、实现）仍然有助于理解和组织神经方法。