[Paper Review] A Trustworthy, Responsible and Interpretable System to Handle Chit Chat in Conversational Bots

Parag Agrawal, Anshuman Suri+1 more|arXiv (Cornell University)|Nov 19, 2018

Misinformation and Its Impacts4 citations

TL;DR

This paper proposes a scalable, interpretable, and trustworthy pipeline for handling chitchat in professional conversational bots by combining hierarchical intent classification, rule-based pattern matching, and a safety-aware sequence-to-sequence generation system. The approach achieves 82% unweighted precision and 79% unweighted recall on chitchat intent detection, significantly outperforming existing systems while ensuring safe, relevant responses through layered filtering and intent prioritization.

ABSTRACT

Most often, chat-bots are built to solve the purpose of a search engine or a human assistant: Their primary goal is to provide information to the user or help them complete a task. However, these chat-bots are incapable of responding to unscripted queries like "Hi, what's up", "What's your favourite food". Human evaluation judgments show that 4 humans come to a consensus on the intent of a given query which is from chat domain only 77% of the time, thus making it evident how non-trivial this task is. In our work, we show why it is difficult to break the chitchat space into clearly defined intents. We propose a system to handle this task in chat-bots, keeping in mind scalability, interpretability, appropriateness, trustworthiness, relevance and coverage. Our work introduces a pipeline for query understanding in chitchat using hierarchical intents as well as a way to use seq-seq auto-generation models in professional bots. We explore an interpretable model for chat domain detection and also show how various components such as adult/offensive classification, grammars/regex patterns, curated personality based responses, generic guided evasive responses and response generation models can be combined in a scalable way to solve this problem.

Motivation & Objective

• To address the challenge of handling unscripted, non-task-oriented user queries in professional chatbots, which often lead to frustration due to generic 'I don't know' responses.
• To design a scalable and interpretable system that can detect chitchat intent with high precision and coverage while ensuring response safety and trustworthiness.
• To reduce ambiguity in chitchat intent classification by introducing a hierarchical intent structure with specific and generic intent levels, minimizing human annotation burden.
• To provide a reliable signal for safe deployment of sequence-to-sequence generation models in professional environments, avoiding offensive or inappropriate responses.
• To improve user engagement by enabling bots to respond naturally to casual queries while maintaining control over response quality and appropriateness.

Proposed method

• The system uses a multi-stage pipeline: first detecting if a query is chitchat via a binary classifier, then classifying it into hierarchical intents (specific and generic) using pattern matching, fuzzy matching, and a neural intent classifier.
• Specific intents are detected via exact, pattern, and fuzzy matching on curated query templates and grammar rules, ensuring high precision (91–98%).
• Generic intents are predicted using a 453-dimensional feature vector fed into a two-layer fully connected neural network with Sigmoid activation, trained on clusters with ≥1,000 queries each.
• A final aggregator component combines specific and generic intent predictions, applying rules to suppress unsafe or conflicting predictions (e.g., discarding specific intents if a criticism generic intent is highly confident).
• The system integrates a safety signal to block auto-generation for risky queries, ensuring that only safe, appropriate queries are passed to sequence-to-sequence models.
• The pipeline combines curated responses, personality-based replies, and generic evasive responses to maintain engagement while minimizing risk.

Experimental results

Research questions

• RQ1 How can chitchat queries be reliably detected and classified in professional conversational bots despite high ambiguity and overlap in intent?
• RQ2 What is the optimal balance between rule-based, pattern-matching, and learned models for scalable and interpretable chitchat intent detection?
• RQ3 How can sequence-to-sequence generation models be safely integrated into professional chatbots without risking offensive or inappropriate responses?
• RQ4 To what extent can hierarchical intent classification (specific vs. generic) improve coverage and precision in chitchat understanding?
• RQ5 Can a modular, rule-informed pipeline outperform end-to-end models in terms of interpretability, safety, and performance on real-world chitchat queries?

Key findings

• The system achieves 82% unweighted precision and 79% unweighted recall on chitchat intent detection, outperforming existing systems (73% precision, 56% recall).
• Specific intent detection via exact and pattern matching achieves 98% and 94% precision respectively, with fuzzy matching at 91% precision, demonstrating high reliability for curated queries.
• The generic intent classifier achieves 78% unweighted precision, which is acceptable given the inherent ambiguity and evasive nature of generic chitchat responses.
• The overall system achieves 100% unweighted coverage across all components, with weighted coverage reaching 100% due to the combination of specific and generic intent detection.
• The safety-aware aggregation layer successfully suppresses risky responses—e.g., discarding a 'command_joke' match when the user is criticizing the bot—improving response trustworthiness.
• The integration of a safety signal enables the use of auto-generation models only for safe queries, reducing the risk of inappropriate responses in professional settings.