[论文解读] Conversational AI: The Science Behind the Alexa Prize

Ashwin Ram, Rohit Prasad+16 more|arXiv (Cornell University)|Jan 11, 2018

Topic Modeling参考文献 18被引用 201

一句话总结

该论文描述了 Alexa Prize，一个 2.5-million-dollar 的大学竞赛，16 支队伍构建社交机器人与人类对话 20 分钟，允许实时数据收集和实时评估，以推动对话式 AI 的发展。

ABSTRACT

Conversational agents are exploding in popularity. However, much work remains in the area of social conversation as well as free-form conversation over a broad range of domains and topics. To advance the state of the art in conversational AI, Amazon launched the Alexa Prize, a 2.5-million-dollar university competition where sixteen selected university teams were challenged to build conversational agents, known as socialbots, to converse coherently and engagingly with humans on popular topics such as Sports, Politics, Entertainment, Fashion and Technology for 20 minutes. The Alexa Prize offers the academic community a unique opportunity to perform research with a live system used by millions of users. The competition provided university teams with real user conversational data at scale, along with the user-provided ratings and feedback augmented with annotations by the Alexa team. This enabled teams to effectively iterate and make improvements throughout the competition while being evaluated in real-time through live user interactions. To build their socialbots, university teams combined state-of-the-art techniques with novel strategies in the areas of Natural Language Understanding, Context Modeling, Dialog Management, Response Generation, and Knowledge Acquisition. To support the efforts of participating teams, the Alexa Prize team made significant scientific and engineering investments to build and improve Conversational Speech Recognition, Topic Tracking, Dialog Evaluation, Voice User Experience, and tools for traffic management and scalability. This paper outlines the advances created by the university teams as well as the Alexa Prize team to achieve the common goal of solving the problem of Conversational AI.

研究动机与目标

• 通过大规模、真实用户评估设定来推动对话式 AI 的研究。
• 描述 Alexa Prize 的架构、数据收集与评估框架，由参赛队伍与组委会共同使用。
• 总结在提升语音识别、主题跟踪、对话评估和对话系统用户体验方面所做的科学与工程投入。

提出的方法

• 描述比赛设计以及对社交机器人（socialbots）的评估标准。
• 解释参赛队伍如何在 NLU、上下文建模、对话管理、应答生成和知识获取等领域，将最先进技术与新颖策略相结合。
• 突出数据收集流程：实时用户对话、评分、反馈，以及 Alexa 团队的标注。
• 讨论在大规模条件下用于对话语音识别、主题跟踪与流量管理的基础设施投资。

实验结果

研究问题

• RQ1 在跨越多样主题的人类对话中，哪些方法能够实现连贯且有吸引力的长上下文社交机器人对话？
• RQ2 在大规模下实时用户反馈如何加速社交机器人能力的迭代改进？
• RQ3 在竞争激烈、真实世界环境中推动对话式 AI 提升的核心研究领域（NLU、上下文、对话、知识）有哪些？
• RQ4 在开展大规模、真实用户参与的 AI 竞赛时，会出现哪些基础设施与方法学方面的挑战？

主要发现

• Alexa Prize 促进了真实用户互动和经过标注的反馈，团队据此迭代社交机器人设计。
• 比赛在核心 AI 领域将最先进技术与新颖策略相结合，推动对话式 AI 的发展。
• 在大规模条件下显著提升语音识别、主题跟踪、对话评估和用户体验，投入了大量科学与工程资源。
• 参赛队伍与组织者利用实时数据对会话代理进行实时评估和优化。