[Paper Review] Fuzzy Model on Human Emotions Recognition

Kaveh Bakhtiyari, Hafizah Husain|arXiv (Cornell University)|Jan 1, 2013

Emotion and Mood Recognition7 references19 citations

TL;DR

This paper proposes a fuzzy logic-based model for multi-level human emotion recognition using keyboard, mouse, and touch-screen interactions, enabling simultaneous detection of multiple emotions at five intensity levels (0–4). Trained with SVM, the system achieves up to 5% higher detection accuracy and a maximum 16.7% false positive rate in emotion level estimation, offering a more natural, continuous representation of human emotions compared to discrete emotion classification methods.

ABSTRACT

This paper discusses a fuzzy model for multi-level human emotions recognition by computer systems through keyboard keystrokes, mouse and touchscreen interactions. This model can also be used to detect the other possible emotions at the time of recognition. Accuracy measurements of human emotions by the fuzzy model are discussed through two methods; the first is accuracy analysis and the second is false positive rate analysis. This fuzzy model detects more emotions, but on the other hand, for some of emotions, a lower accuracy was obtained with the comparison with the non-fuzzy human emotions detection methods. This system was trained and tested by Support Vector Machine (SVM) to recognize the users' emotions. Overall, this model represents a closer similarity between human brain detection of emotions and computer systems.

Motivation & Objective

• To address the limitations of discrete emotion detection by modeling emotions as continuous, multi-level states rather than binary (present/absent) classifications.
• To enable simultaneous recognition of multiple emotions and their respective intensity levels, reflecting the complexity of real human emotional states.
• To develop a computationally feasible, non-invasive emotion recognition system using widely available input modalities—keyboard, mouse, and touch-screen—without requiring specialized sensors.
• To improve the naturalness and accuracy of human-computer interaction by emulating the brain's fuzzy-based emotional processing.

Proposed method

• Emotions are classified on a 5-level fuzzy scale (0 = none, 4 = maximum intensity), enabling continuous emotion representation.
• Experience Sampling Methodology (ESM) was used to collect self-reported emotional states from 130 participants across diverse cultural backgrounds.
• The PANAS framework was extended by adding 7 basic emotions, resulting in a 27-emotion dataset for training and evaluation.
• Support Vector Machine (SVM) was employed as the classification engine to map behavioral features (keystroke dynamics, mouse movements, touch interactions) to fuzzy emotion levels.
• Fuzzy logic was applied to model probabilistic co-occurrence of emotions, allowing estimation of secondary or co-existing emotions at given intensity levels.
• Evaluation used two metrics: (1) correct emotion name detection (accuracy), and (2) correct level estimation with low false positive rate.

Experimental results

Research questions

• RQ1 Can a fuzzy model effectively represent human emotions as continuous intensity levels (0–4) rather than discrete categories?
• RQ2 To what extent can a system detect multiple co-occurring emotions simultaneously with high accuracy and low false positive rates?
• RQ3 How do cultural and demographic factors (e.g., region, gender) influence the patterns of emotion co-occurrence and intensity in behavioral data?
• RQ4 Does using fuzzy emotion representation improve the realism and accuracy of emotion recognition compared to traditional binary or single-emotion detection models?

Key findings

• The fuzzy model achieved up to 5% higher detection accuracy compared to non-fuzzy methods by recognizing multiple emotions and their intensity levels.
• The false positive rate for emotion level estimation ranged from 0% to 16.7%, indicating reliable estimation of intensity across different emotional states.
• Emotions such as Joy, Anticipation, Anger, and Fear showed distinct but overlapping change patterns, with Disgust and Anger sharing similar dynamic profiles.
• The model successfully estimated co-existing emotions—e.g., when Joy was at 50%, Fear could be estimated at 26% and Acceptance at 55%, demonstrating multi-emotion awareness.
• Men reported stronger self-reported emotions overall, with 'Active' being the strongest (3.24/4) and 'Unfriendly' the weakest (1.62/4).
• Regional differences were observed: Middle Eastern participants reported higher levels of Fear (1.33) and Surprise (1.33) at 50% Joy, while Europeans reported higher Acceptance (2.09) and Anticipation (1.8) under the same condition.