[Paper Review] Controlling Style in Generated Dialogue

Eric M. Smith, Diana Gonzalez-Rico+2 more|arXiv (Cornell University)|Sep 22, 2020

Topic Modeling29 references34 citations

TL;DR

The paper adapts three controllable generation methods to open-domain dialogue to control style across about 217 target styles, comparing retrieve-and-style-transfer, plug-and-play refinement, and conditioned generation on state-of-the-art conversational models. Fine-tuned conditioned generation performs best for matching target styles with better inference speed.

ABSTRACT

Open-domain conversation models have become good at generating natural-sounding dialogue, using very large architectures with billions of trainable parameters. The vast training data required to train these architectures aggregates many different styles, tones, and qualities. Using that data to train a single model makes it difficult to use the model as a consistent conversational agent, e.g. with a stable set of persona traits and a typical style of expression. Several architectures affording control mechanisms over generation architectures have been proposed, each with different trade-offs. However, it remains unclear whether their use in dialogue is viable, and what the trade-offs look like with the most recent state-of-the-art conversational architectures. In this work, we adapt three previously proposed controllable generation architectures to open-domain dialogue generation, controlling the style of the generation to match one among about 200 possible styles. We compare their respective performance and tradeoffs, and show how they can be used to provide insights into existing conversational datasets, and generate a varied set of styled conversation replies.

Motivation & Objective

• Motivate the need for stable persona and style in open-domain dialogue models trained on diverse data.
• Evaluate three controllable generation approaches for dialogue: retrieve-and-style-transfer, plug-and-play refinement, and conditioned generation.
• Quantify trade-offs in style control accuracy, training/inference cost, and impact on other conversational metrics.
• Demonstrate a practical pipeline to label in-domain data for style conditioning and validate generalization to new style spaces.

Proposed method

• Adapt three controllable generation architectures to dialogue: retrieve-and-style-transfer (RnST), plug-and-play language model (PPLM) refinement, and conditioned generation (C).
• Use a mid-size 217-style space from Image-Chat to train and evaluate style-control classifiers and generators.
• Fine-tune generative models on D+ (D augmented with inferred or provided style labels) to enable style-conditioned generation.
• Compare performance in terms of automated style-control accuracy, human judgments, and conversational quality metrics.
• Assess resource costs (training/inference) and practical deployment considerations for each approach.

Experimental results

Research questions

• RQ1 Can open-domain dialogue models reliably generate with a specified target style across a large style space?
• RQ2 What are the trade-offs between retrieve-and-transfer, iterative refinement, and conditioned generation for style control in dialogue?
• RQ3 How well does style transfer generalize from image-grounded style labels to purely textual dialogue?
• RQ4 Can a practical pipeline label unlabelled dialogue data to enable effective style-conditioned fine-tuning?
• RQ5 How do style-control methods affect core dialogue metrics such as relevance and fluency?

Key findings

• Fine-tuned conditioned generation (C) yields strongest performance in matching target styles and preserves other conversational metrics better than alternatives.
• Retrieve-and-style-transfer (RnST) with style conditioning on the retrieved context provides weaker style control than pure generation, and can degrade style accuracy.
• PPLM-based refinement offers flexible, fine-grained control but is computationally costly at inference, with limited gains for the large 217-style space used here.
• A practical labeling approach (D+), using a classifier trained on a style-labelled dataset to label unlabeled data, enables effective style-conditioned fine-tuning on standard dialogue datasets.
• The best performing model (C100/C75) achieves meaningful style control with tabled style-accuracy improvements and reasonable perplexities, and infers inferences efficiently compared to iterative refinement.
• The study provides a generalizable pipeline to apply style conditioning to any style space for which a classifier can be trained.