[Paper Review] Social Robot Modelling of Human Affective State

David B. Skillicorn, Nasser A Alsadhan+2 more|arXiv (Cornell University)|May 2, 2017

Mental Health Research Topics2 references3 citations

TL;DR

This paper proposes a computational framework for social robots to infer human affective states—emotions, mood, personality, desires, and attitudes—from short, informal text like tweets and Facebook posts using NLP techniques. Despite limited and noisy input, the model achieves strong performance, particularly in personality prediction (70% accuracy), demonstrating that lightweight, real-time affective modeling is feasible for social robotics.

ABSTRACT

Social robots need to understand the affective state of the humans with whom they interact. Successful interactions require understanding mood and emotion in the short term, and personality and attitudes over longer periods. Social robots should also be able to infer the desires, wishes, and preferences of humans without being explicitly told. We investigate how effectively affective state can be inferred from corpora in which documents are plausible surrogates for what a robot might hear. For mood, emotions, wishes, desires, and attitudes we show highly ranked documents; for personality dimensions, estimates of ground truth are available and we report performance accuracy. The results are surprisingly strong given the limited information in short documents.

Motivation & Objective

• To enable social robots to model human affective states—emotions, mood, personality, desires, and attitudes—without explicit instruction.
• To investigate whether short, informal texts (e.g., tweets, Facebook posts) can serve as reliable surrogates for real-time human-robot interaction data.
• To develop a lightweight, computationally efficient method for affective state inference using minimal textual input.
• To evaluate the performance of affective state modeling across different components of affect, especially where ground truth is available (e.g., personality).
• To determine the feasibility of real-time affective modeling in social robots using practical computational resources.

Proposed method

• The study uses tweets, headlines, and Facebook posts as surrogates for real-time human speech, treating them as data streams a robot might overheard.
• For emotion and attitude detection, the model ranks documents based on affective content using a modified Osgood semantic differential model.
• Personality inference is performed using a matrix factorization approach based on the OCEAN model (Openness, Conscientiousness, Extraversion, Agreeableness, Neuroticism), with ground truth available for validation.
• The method applies singular value decomposition (SVD) to infer latent affective dimensions from text, with a technique to avoid recomputing SVD when new test data is added.
• The model uses the Moore-Penrose inverse to efficiently update the transformation matrix $\hat{V}$ when new rows (documents) are appended to the data matrix $\hat{A}$.
• Face validation is used for emotions, desires, and attitudes: high-ranked documents are manually inspected to confirm they exhibit the target affective property.

Experimental results

Research questions

• RQ1 Can short, informal texts such as tweets and Facebook posts reliably convey information about human emotions, mood, and personality?
• RQ2 How accurately can a computational model infer personality traits (OCEAN) from minimal textual input?
• RQ3 To what extent can a robot infer desires, wishes, and preferences from unstructured, informal text without explicit commands?
• RQ4 How effective is the Osgood model in detecting weak attitudes and emotional valence in short, noisy text?
• RQ5 Can lightweight, real-time affective modeling be achieved with minimal computational overhead in social robots?

Key findings

• The model achieves 70% accuracy in predicting personality traits (OCEAN) from short texts, outperforming typical human performance (80% accuracy) in similar tasks.
• Despite limited and noisy input, the model successfully identifies high-ranking documents that exhibit specific emotions, moods, and attitudes, as confirmed by face validation.
• The Osgood model shows weaker performance in detecting attitudes, likely due to the diffuse and low-weighted nature of affective expressions in short texts.
• The method enables efficient incremental updates to the SVD-based model, avoiding full recomputation and supporting real-time inference.
• The results demonstrate that even minimal textual input can support usable affective state modeling, suggesting feasibility for real-world social robot applications.
• The study confirms that mood significantly modulates emotional expression, reinforcing the need for mood modeling as a prerequisite for accurate emotion inference.