[Paper Review] Controllable Mixed-Initiative Dialogue Generation through Prompting

Maximillian Chen, Xiao Yu+3 more|arXiv (Cornell University)|May 6, 2023

Topic Modeling4 citations

TL;DR

This paper proposes using few-shot prompting with large language models (LLMs) as a drop-in replacement for fine-tuning in controllable mixed-initiative dialogue generation. It demonstrates that prompt-based LLMs outperform both fine-tuned models and ground-truth responses in human evaluations and automatic metrics on two tasks: Emotional Support Conversations and PersuasionForGood.

ABSTRACT

Mixed-initiative dialogue tasks involve repeated exchanges of information and conversational control. Conversational agents gain control by generating responses that follow particular dialogue intents or strategies, prescribed by a policy planner. The standard approach has been fine-tuning pre-trained language models to perform generation conditioned on these intents. However, these supervised generation models are limited by the cost and quality of data annotation. We instead prompt large language models as a drop-in replacement to fine-tuning on conditional generation. We formalize prompt construction for controllable mixed-initiative dialogue. Our findings show improvements over fine-tuning and ground truth responses according to human evaluation and automatic metrics for two tasks: PersuasionForGood and Emotional Support Conversations.

Motivation & Objective

• To address the limitations of fine-tuned models in controllable dialogue generation, such as high annotation costs and sensitivity to data quality and distribution shifts.
• To explore whether few-shot prompting of pre-trained LLMs can achieve better control and response quality than supervised fine-tuning.
• To evaluate the effectiveness of prompt engineering in enabling LLMs to generate responses aligned with specific dialogue strategies in mixed-initiative settings.
• To compare prompt-based LLMs against fine-tuned models and human-annotated ground truth responses in both automatic and human evaluation metrics.

Proposed method

• Formalizing prompt construction for controllable mixed-initiative dialogue by designing structured few-shot prompts that condition LLMs on dialogue history and desired strategy.
• Using in-context learning with few-shot demonstrations to guide the LLM to generate responses matching specific dialogue strategies (e.g., restatement, empathy, redirection).
• Leveraging large pre-trained language models like GPT-3 as a zero-shot or few-shot generation head, replacing the need for task-specific fine-tuning.
• Designing prompts that include the dialogue history, the target strategy, and example response pairs to guide the model’s output.
• Evaluating the system on two benchmark datasets: Emotional Support Conversations (ESC) and PersuasionForGood (P4G), using both automatic metrics and human evaluation.
• Comparing the prompt-based LLM approach against fine-tuned models (e.g., Oracle-BlenderBot) and human-annotated ground truth responses.

Experimental results

Research questions

• RQ1 Can few-shot prompting of large language models effectively generate responses aligned with specific dialogue strategies in mixed-initiative settings?
• RQ2 How does prompt-based LLM generation compare to fine-tuned models in terms of response quality and strategy adherence?
• RQ3 Does prompting with in-context examples improve generation quality over fine-tuning when training data is noisy or distributionally shifted?
• RQ4 Can prompt-based LLMs achieve human-level response quality and strategy control without task-specific fine-tuning?
• RQ5 How robust is the prompt-based approach to variations in user input, including low-resource or non-fluent language patterns?

Key findings

• The prompt-based LLM approach achieved superior performance in human evaluations compared to both fine-tuned models and ground-truth responses on the Emotional Support Conversations dataset.
• On the PersuasionForGood dataset, the prompt-based LLM outperformed fine-tuned models in both automatic metrics and human evaluations, particularly in perceived persuasiveness and coherence.
• Human evaluators rated the prompt-based LLM responses as more natural, consistent, and coherent than those from fine-tuned models, even when the latter were trained on ground-truth strategy labels.
• The approach demonstrated robustness to distributional shift and annotation noise, as the LLMs generalized better across diverse conversational contexts without fine-tuning.
• In cases of non-fluent or low-resource user inputs, the prompt-based LLM maintained response quality and coherence, outperforming fine-tuned models that struggled with out-of-distribution inputs.
• The study found that prompting with carefully designed in-context examples enabled LLMs to effectively emulate complex dialogue strategies such as restatement, empathy, and redirection with high fidelity.