[Paper Review] A Taxonomy of Empathetic Response Intents in Human Social Conversations

Anuradha Welivita, Pearl Pu|arXiv (Cornell University)|Dec 7, 2020

Topic Modeling50 references4 citations

TL;DR

This paper proposes a taxonomy of 500 empathetic listener response intents derived from human social conversations in the EmpatheticDialogues dataset, using manual annotation and automated labeling to identify patterns between speaker emotions and listener responses. The key contribution is a structured, interpretable framework that enhances chatbot controllability and empathy by integrating intent and emotion modeling into neural response generation, revealing that listeners frequently use questioning rather than emotional mirroring to respond to sadness or anger.

ABSTRACT

Open-domain conversational agents or chatbots are becoming increasingly popular in the natural language processing community. One of the challenges is enabling them to converse in an empathetic manner. Current neural response generation methods rely solely on end-to-end learning from large scale conversation data to generate dialogues. This approach can produce socially unacceptable responses due to the lack of large-scale quality data used to train the neural models. However, recent work has shown the promise of combining dialogue act/intent modelling and neural response generation. This hybrid method improves the response quality of chatbots and makes them more controllable and interpretable. A key element in dialog intent modelling is the development of a taxonomy. Inspired by this idea, we have manually labeled 500 response intents using a subset of a sizeable empathetic dialogue dataset (25K dialogues). Our goal is to produce a large-scale taxonomy for empathetic response intents. Furthermore, using lexical and machine learning methods, we automatically analysed both speaker and listener utterances of the entire dataset with identified response intents and 32 emotion categories. Finally, we use information visualization methods to summarize emotional dialogue exchange patterns and their temporal progression. These results reveal novel and important empathy patterns in human-human open-domain conversations and can serve as heuristics for hybrid approaches.

Motivation & Objective

• To develop a large-scale, manually curated taxonomy of empathetic listener response intents in human social conversations to improve chatbot empathy.
• To address the limitations of purely data-driven neural response generation, which often produces socially inappropriate or redundant responses.
• To analyze the relationship between speaker emotions and listener response intents using automatic emotion and intent labeling across 25,000 dialogues.
• To visualize temporal patterns in emotional dialogue exchanges to uncover recurring empathy dynamics in open-domain human conversations.
• To provide a reusable annotation scheme and heuristics for training more controllable, interpretable, and prosocial empathetic chatbots.

Proposed method

• Manually annotated 500 response utterances from the EmpatheticDialogues dataset using a custom taxonomy of empathetic listener intents across 32 emotion categories.
• Applied lexical pattern matching and machine learning to automatically label all 25,000 dialogues in the dataset with both emotion and intent categories.
• Used information visualization techniques to map and analyze emotional dialogue exchange patterns and their evolution over time.
• Proposed a hybrid framework where dialogue history is tagged with emotion and intent, and a dialogue act selection module chooses the most appropriate listener intent before response generation.
• Integrated the taxonomy into a joint modeling approach that conditions response generation on both conversation history and selected empathetic intent labels.
• Evaluated the robustness of automatic labeling by comparing lexical-based intent detection with context-aware improvements, acknowledging limitations in sentence-level classification without dialogue context.

Experimental results

Research questions

• RQ1 What are the most frequent and distinct empathetic response intents used by humans in open-domain social conversations?
• RQ2 How do listener response intents correlate with the speaker’s expressed emotions, particularly in cases of sadness, anger, or disgust?
• RQ3 What are the dominant emotional and conversational patterns in the temporal progression of empathetic dialogue exchanges?
• RQ4 To what extent can automatic labeling of emotions and intents in large-scale dialogue datasets support the development of controllable empathetic chatbots?
• RQ5 How can a taxonomy of empathetic intents improve the interpretability and controllability of neural response generation models?

Key findings

• Listeners are significantly more likely to respond to sad or angry emotions with questioning rather than by expressing similar or opposite emotions, indicating a strategic use of inquiry to support the speaker.
• Personal distress—where listeners experience emotional contagion and react with their own negative emotions—occurs but is less frequent than healthy empathetic responses, suggesting a natural tendency toward prosocial engagement.
• The most common empathetic response strategies involve active listening, clarification, and validation, rather than emotional mirroring or self-revelation.
• The taxonomy successfully captures nuanced empathetic behaviors such as compassion, concern, and supportive questioning, distinguishing them from unhelpful or distressing reactions.
• Visualization of emotional progression reveals that empathy patterns evolve dynamically over dialogue turns, with early responses often being more reactive and later ones more reflective.
• The automatic labeling pipeline, while effective at scale, shows limitations in distinguishing context-dependent intent variations, especially when relying solely on sentence-level features without dialogue history.