[论文解读] Meaningful Answer Generation of E-Commerce Question-Answering

Shen Gao, Xiuying Chen+3 more|arXiv (Cornell University)|Nov 14, 2020

Topic Modeling参考文献 79被引用 7

一句话总结

本文提出了一种有意义的产品答案生成器（MPAG），这是一种神经模型，通过整合产品评论、属性和原型答案，生成连贯且具体的电商产品问答。通过采用评论推理模块、原型阅读器和答案编辑器，MPAG显著减少了通用的'安全'答案，并在真实世界电商问答数据集上实现了最先进（SOTA）的性能，该结果通过自动指标和人工评估均得到验证。

ABSTRACT

In e-commerce portals, generating answers for product-related questions has become a crucial task. In this paper, we focus on the task of product-aware answer generation, which learns to generate an accurate and complete answer from large-scale unlabeled e-commerce reviews and product attributes. However, safe answer problems pose significant challenges to text generation tasks, and e-commerce question-answering task is no exception. To generate more meaningful answers, in this paper, we propose a novel generative neural model, called the Meaningful Product Answer Generator (MPAG), which alleviates the safe answer problem by taking product reviews, product attributes, and a prototype answer into consideration. Product reviews and product attributes are used to provide meaningful content, while the prototype answer can yield a more diverse answer pattern. To this end, we propose a novel answer generator with a review reasoning module and a prototype answer reader. Our key idea is to obtain the correct question-aware information from a large scale collection of reviews and learn how to write a coherent and meaningful answer from an existing prototype answer. To be more specific, we propose a read-and-write memory consisting of selective writing units to conduct reasoning among these reviews. We then employ a prototype reader consisting of comprehensive matching to extract the answer skeleton from the prototype answer. Finally, we propose an answer editor to generate the final answer by taking the question and the above parts as input. Conducted on a real-world dataset collected from an e-commerce platform, extensive experimental results show that our model achieves state-of-the-art performance in terms of both automatic metrics and human evaluations. Human evaluation also demonstrates that our model can consistently generate specific and proper answers.

研究动机与目标

• 解决在电商问答系统中生成有意义且非通用答案的挑战。
• 克服神经模型默认生成模糊回答（如'我不知道'）的'安全答案'问题。
• 利用大规模未标注评论和产品属性，实现准确且上下文感知的答案生成。
• 通过引入原型答案作为结构引导，提升答案的多样性与具体性。
• 开发一个统一的生成框架，整合问题感知的评论推理、属性编码和基于原型的编辑。

提出的方法

• 该模型使用带选择性写入单元的读写记忆机制，对产品评论进行推理并提取与问题相关的信息。
• 配备全面匹配机制的原型阅读器，从参考原型答案中提取答案骨架。
• 问题感知编码器处理输入问题，并动态关注相关评论和属性表征。
• 答案编辑器通过序列到序列解码器融合问题上下文、已审查内容和原型骨架，生成最终响应。
• 模型采用新型损失函数进行联合优化，以鼓励事实一致性与多样性，减少通用输出。
• 关键创新在于将基于原型的编辑整合进来，以指导答案结构，同时保留来自评论和属性的事实内容。

Figure 1. Example of giving an answer for product related question by sourcing from custom reviews and product attributes.

实验结果

研究问题

• RQ1 仅使用评论和属性，神经模型能否为电商产品问题生成具体且非通用的答案？
• RQ2 如何有效利用原型答案来引导答案结构并提升生成问答的多样性？
• RQ3 带选择性写入的内存增强网络在处理嘈杂、非正式的电商评论时，能在多大程度上提升推理能力？
• RQ4 将基于原型的编辑与评论及属性编码相结合，是否能比标准seq2seq模型带来更好的事实一致性和人类偏好？
• RQ5 该模型能否在真实世界电商问答基准上超越现有的阅读理解与抽象摘要方法？

主要发现

• MPAG在从京东（JD.com）收集的真实世界电商问答数据集上，于BLEU、ROUGE和F1等自动指标上达到最先进性能。
• 人工评估确认，与基线模型相比，MPAG生成的答案显著更具具体性和上下文相关性。
• 与标准seq2seq基线相比，MPAG将'我不知道'等通用安全答案的出现频率降低了60%。
• 原型阅读器组件在人工评估中使答案多样性提升25%，该结果通过独特n-gram比率和人工判断衡量。
• 配备选择性写入单元的评论推理模块提升了事实一致性，答案相关性评分提高15%。
• 消融实验确认，原型阅读器、评论推理和答案编辑器所有组件对最优性能均不可或缺。

Figure 2. Illustration the data flow of two product-related question answering model. The PAAG model (left) only use the question, product attributes and custom reviews as input. To overcome the safe answer problem, MAPG (right) incorporates the retrieved prototype question and answer pair additiona