[Paper Review] Think Before You Speak: Using Self-talk to Generate Implicit Commonsense Knowledge for Response Generation

Pei Zhou, Karthik Gopalakrishnan+6 more|arXiv (Cornell University)|Oct 16, 2021

Topic Modeling50 references4 citations

TL;DR

This paper proposes a self-talk approach that uses a single generative model to first produce implicit commonsense knowledge and then generate responses grounded in that knowledge, significantly improving response quality and common sense alignment compared to end-to-end models, with human evaluations showing high-quality knowledge generation in 75% of cases.

ABSTRACT

Implicit knowledge, such as common sense, is key to fluid human conversations. Current neural response generation (RG) models are trained end-to-end, omitting unstated implicit knowledge. In this paper, we present a self-talk approach that first generates the implicit commonsense knowledge and then generates response by referencing the externalized knowledge, all using one generative model. We analyze different choices to collect knowledge-aligned dialogues, represent implicit knowledge, and elicit knowledge and responses. We introduce three evaluation aspects: knowledge quality, knowledge-response connection, and response quality and perform extensive human evaluations. Our experimental results show that compared with end-to-end RG models, self-talk models that externalize the knowledge grounding process by explicitly generating implicit knowledge also produce responses that are more informative, specific, and follow common sense. We also find via human evaluation that self-talk models generate high-quality knowledge around 75% of the time. We hope that our findings encourage further work on different approaches to modeling implicit commonsense knowledge and training knowledgeable RG models.

Motivation & Objective

• To address the lack of implicit commonsense knowledge in end-to-end neural response generation models.
• To explore how externalizing implicit knowledge through self-talk improves response quality and coherence.
• To develop a unified generative model that produces both knowledge and responses in a single training process.
• To evaluate the quality of generated knowledge, its connection to responses, and overall response performance.

Proposed method

• Uses a single generative model to first generate implicit commonsense knowledge and then produce responses based on that knowledge.
• Employs knowledge-aligned dialogue data collected through specific data collection strategies to ensure knowledge and response alignment.
• Represents implicit knowledge as natural language text, externalizing it for model access and reasoning.
• Elicits knowledge and responses through a two-stage generation process within one model architecture.
• Applies human evaluation to assess knowledge quality, knowledge-response connection, and response quality.
• Compares the self-talk model against standard end-to-end response generation models under controlled evaluation metrics.

Experimental results

Research questions

• RQ1 Can a self-talk approach that generates implicit commonsense knowledge improve response quality in open-domain dialogue?
• RQ2 How well does the generated knowledge align with the context and the final response?
• RQ3 What is the quality of the implicit knowledge produced by the model, and how often is it relevant and accurate?
• RQ4 How does the self-talk model compare to end-to-end response generation models in terms of informativeness and common sense adherence?

Key findings

• Self-talk models generate responses that are significantly more informative and specific than end-to-end models.
• The self-talk approach improves alignment with common sense, producing responses that are more contextually appropriate.
• Human evaluations show that the model generates high-quality implicit knowledge approximately 75% of the time.
• The knowledge-response connection is stronger in self-talk models, indicating better grounding of responses in explicit knowledge.
• The model outperforms end-to-end baselines across all three evaluation dimensions: knowledge quality, knowledge-response connection, and response quality.
• Externalizing the knowledge grounding process leads to more reliable and interpretable response generation.