[Paper Review] MoodSwipe: A Soft Keyboard that Suggests Messages Based on User-Specified Emotions

Chieh-Yang Huang, Tristan Labetoulle+5 more|arXiv (Cornell University)|Jul 22, 2017

Multimedia Communication and Technology17 references3 citations

TL;DR

MoodSwipe is a soft keyboard that suggests text messages based on user-specified emotions, using real dialog data to train emotion-aware text suggestion models. By letting users swipe to select emotions and receive contextually appropriate message suggestions, it improves message clarity and comfort—especially for expressing anger—while automatically collecting high-quality, user-reported emotion labels for future model improvement.

ABSTRACT

We present MoodSwipe, a soft keyboard that suggests text messages given the user-specified emotions utilizing the real dialog data. The aim of MoodSwipe is to create a convenient user interface to enjoy the technology of emotion classification and text suggestion, and at the same time to collect labeled data automatically for developing more advanced technologies. While users select the MoodSwipe keyboard, they can type as usual but sense the emotion conveyed by their text and receive suggestions for their message as a benefit. In MoodSwipe, the detected emotions serve as the medium for suggested texts, where viewing the latter is the incentive to correcting the former. We conduct several experiments to show the superiority of the emotion classification models trained on the dialog data, and further to verify good emotion cues are important context for text suggestion.

Motivation & Objective

• To address the challenge of collecting self-reported emotion labels for dialog messages, which are costly and disruptive to collect via traditional methods.
• To close the loop in emotion-sensitive communication by enabling real-time, sender-side support for emotional expression in text-based messaging.
• To explore how user-specified emotions can enhance text suggestion quality, particularly in terms of clarity, comfort, and responsiveness.
• To develop a scalable, interactive interface that incentivizes users to label their own emotions naturally through engagement with message suggestions.
• To collect high-quality, user-reported emotion-labeled dialog data through implicit user actions like swiping and selecting suggestions.

Proposed method

• The MoodSwipe keyboard integrates a standard soft keyboard with a color bar that dynamically reflects the detected emotion of the user’s typed message using a pre-trained emotion classifier.
• Users can swipe across the color bar to specify a target emotion (e.g., Joy, Anger, Sadness), and the system instantly suggests alternative message formulations for that emotion.
• The system uses a retrieval-based model that retrieves relevant responses from a large dialog database, with emotion labels used as filters and context cues to improve suggestion relevance.
• Emotion labels are collected automatically: a 'select' action records the emotion of the chosen suggestion, while a 'swipe' without selection infers the intended emotion from the final color position.
• The emotion classifier is trained on real dialog data, using psychological frameworks to map emotions to colors (e.g., red for anger, blue for sadness) for intuitive user interaction.
• Human evaluation via Amazon Mechanical Turk assesses suggested messages on clarity, comfort, and responsiveness, with results compared against original user inputs and baseline models.

Experimental results

Research questions

• RQ1 Can user-specified emotions significantly improve the quality of text suggestions in a real-time, mobile keyboard interface?
• RQ2 How does incorporating emotion context affect the perceived clarity, comfort, and responsiveness of suggested messages?
• RQ3 Can implicit user actions like swiping and selecting be reliably used to collect high-quality, self-reported emotion labels for dialog messages?
• RQ4 Does the inclusion of emotion signals lead to better suggestions than baseline retrieval models without emotion context?
• RQ5 Are certain emotions (e.g., anger) more amenable to effective suggestion than others, and why?

Key findings

• The MoodSwipe system achieved a Good Suggestion Rate of 28.65% for comfort in suggested messages when emotion context was used, compared to 28.38% in the baseline (without emotion), indicating a slight but consistent improvement.
• For anger-related messages, the Good Suggestion Rate reached 37.49% in the [+Emotion] setting, significantly higher than the average rate of 28%, showing particular effectiveness for expressing anger.
• The system successfully collected 1,366 user-reported emotion-labeled dialog messages through user interactions, demonstrating the feasibility of automatic labeling via swiping and selecting.
• Human evaluations showed that suggested messages were ranked as better (lower rank) than original user inputs in 26% to 28% of cases, indicating practical utility.
• The addition of emotion context led to slightly better average rankings across all three evaluation dimensions—clarity, comfort, and responsiveness—confirming its value in message suggestion.
• The results suggest that emotion-aware text suggestion is most effective for emotionally charged messages, particularly anger, where users benefit most from alternative phrasing.