[Paper Review] Recipes for Safety in Open-domain Chatbots

Jing Xu, Da Young Ju+4 more|arXiv (Cornell University)|Oct 14, 2020

Hate Speech and Cyberbullying DetectionComputer Science68 references98 citations

TL;DR

The paper proposes a human-and-model-in-the-loop framework and a baked-in safety approach to reduce toxic behavior in open-domain chatbots, comparing multiple safety strategies and evaluating safety vs. engagingness. It also introduces Bot-Adversarial Dialogue Safety (BAD) and analyzes their effectiveness against existing methods.

ABSTRACT

Models trained on large unlabeled corpora of human interactions will learn patterns and mimic behaviors therein, which include offensive or otherwise toxic behavior and unwanted biases. We investigate a variety of methods to mitigate these issues in the context of open-domain generative dialogue models. We introduce a new human-and-model-in-the-loop framework for both training safer models and for evaluating them, as well as a novel method to distill safety considerations inside generative models without the use of an external classifier at deployment time. We conduct experiments comparing these methods and find our new techniques are (i) safer than existing models as measured by automatic and human evaluations while (ii) maintaining usability metrics such as engagingness relative to the state of the art. We then discuss the limitations of this work by analyzing failure cases of our models.

Motivation & Objective

• Investigate how to mitigate offensive, biased, and unsafe behavior in open-domain dialogue models trained on large human interaction corpora.
• Compare unsafe utterance detection, safe utterance generation, sensitive-topic avoidance, and gender-bias mitigation across two-stage and end-to-end approaches.
• Introduce novel safety methods (Bot-Adversarial Dialogue Safety and Baking-in Safety) and assess trade-offs between safety and engagingness.

Proposed method

• Evaluate Unsafe Utterance Detection and its deployment in two-stage models (safety classifier as a layer).
• Develop Safe Utterance Generation techniques including data pre-processing, safe beam blocking, safety and style control, and baking-in safety.
• Explore Sensitive Topic Avoidance and Gender Bias Mitigation as complementary safety strategies.
• Introduce Bot-Adversarial Dialogue Safety (BAD) data collection where humans adversarially probe bots to elicit unsafe responses to improve classifiers.
• Distill safety considerations into generative models so deployment no longer requires an external safety classifier.

Experimental results

Research questions

• RQ1 Can safety classifiers robustly detect unsafe content in multi-turn open-domain dialogues?
• RQ2 Do end-to-end safe-generation approaches (including baking-in safety) outperform two-stage safety pipelines in real-world engagement and safety metrics?
• RQ3 What is the impact of sensitive-topic avoidance and gender-bias mitigation on safety and engagingness?
• RQ4 How does Bot-Adversarial Dialogue Safety improve robustness of safety models compared to existing classifiers?

Key findings

• Two-stage safety models with detectors can outperform existing safety classifiers in safety, while maintaining engagingness.
• Baked-in safety models, where safety is distilled into the generator during training, can reduce the need for external classifiers at test time and improve safety with preserved engagement.
• Data baking and safety-aware training can mitigate unsafe outputs even when the model is exposed to toxic pre-training data.
• BOT-Adversarial Dialogue Safety data collection yields large amounts of adversarial dialogue data (e.g., 5k conversations and ~70k utterances) to train more robust safety classifiers.
• Data filtering and control-based strategies can influence safety and style without severely compromising engagingness.