[论文解读] Does the Goal Matter? Emotion Recognition Tasks Can Change the Social Value of Facial Mimicry towards Artificial Agents

Giulia Perugia, Maike Paetzel+5 more|arXiv (Cornell University)|Oct 29, 2021

Face Recognition and Perception参考文献 73被引用 8

一句话总结

本研究探讨了人工代理的具身性与类人程度如何影响情绪识别任务中自发性与指令性面部模仿。通过计算机视觉检测面部动作单元强度，研究发现类人程度越高的代理，其自发模仿越少，表明在此类情境下，模仿可能服务于识别而非亲密度。

ABSTRACT

In this paper, we present a study aimed at understanding whether the embodiment and humanlikeness of an artificial agent can affect people's spontaneous and instructed mimicry of its facial expressions. The study followed a mixed experimental design and revolved around an emotion recognition task. Participants were randomly assigned to one level of humanlikeness (between-subject variable: humanlike, characterlike, or morph facial texture of the artificial agents) and observed the facial expressions displayed by three artificial agents differing in embodiment (within-subject variable: video-recorded robot, physical robot, and virtual agent) and a human (control). To study both spontaneous and instructed facial mimicry, we divided the experimental sessions into two phases. In the first phase, we asked participants to observe and recognize the emotions displayed by the agents. In the second phase, we asked them to look at the agents' facial expressions, replicate their dynamics as closely as possible, and then identify the observed emotions. In both cases, we assessed participants' facial expressions with an automated Action Unit (AU) intensity detector. Contrary to our hypotheses, our results disclose that the agent that was perceived as the least uncanny, and most anthropomorphic, likable, and co-present, was the one spontaneously mimicked the least. Moreover, they show that instructed facial mimicry negatively predicts spontaneous facial mimicry. Further exploratory analyses revealed that spontaneous facial mimicry appeared when participants were less certain of the emotion they recognized. Hence, we postulate that an emotion recognition goal can flip the social value of facial mimicry as it transforms a likable artificial agent into a distractor. Further work is needed to corroborate this hypothesis. Nevertheless, our findings shed light on the functioning of human-agent and human-robot mimicry in emotion recognition tasks and help us to unravel the relationship between facial mimicry, liking, and rapport.

研究动机与目标

• 检验人工代理的具身性与类人程度是否会影响人类参与者的自发性与指令性面部模仿。
• 探究面部模仿是否可作为人机交互中喜爱与亲密度的潜在行为指标。
• 探索任务情境（特别是情绪识别）如何影响对人工代理面部模仿的社会价值。
• 评估使用自动化面部动作单元检测作为人机交互场景中非侵入性行为测量方法的可行性。
• 挑战模仿总是传递积极社会评价的假设，尤其是在人工代理情境中。

提出的方法

• 采用混合实验设计，45名参与者参与情绪识别任务，涉及三种人工代理（实体机器人、视频录制的机器人、虚拟代理）和一个人类对照组。
• 将代理类人程度分为三个水平：类人、角色类、变形面部纹理（组内变量）。
• 将实验分为两个阶段：(1) 情绪识别过程中的自发模仿；(2) 指令性模仿，复制面部动态。
• 使用Hupont与Chetouani的基于计算机视觉的面部动作单元强度检测器，实时分析参与者面部表情。
• 在第二阶段重新排列刺激顺序，以模拟有意模仿，基于模仿可提高情绪识别准确性的假设。
• 收集并分析视频数据，通过AU强度评分量化自发性与指令性面部模仿。

实验结果

研究问题

• RQ1 人工代理的类人程度与具身性是否会影响人类参与者自发面部模仿的频率？
• RQ2 在情绪识别任务中，指令性面部模仿与自发面部模仿的相关性如何？
• RQ3 存在目标（情绪识别）是否改变了对人工代理面部模仿的社会价值？
• RQ4 面部模仿能否作为人机交互中喜爱与亲密度的可靠行为指标？
• RQ5 在何种条件下，模仿更可能作为识别辅助而非社交联结机制发挥作用？

主要发现

• 最具类人特征、最讨人喜欢且共在的代理——被感知为最不诡异——在自发模仿中被模仿得最少，与初始假设相反。
• 指令性面部模仿与自发面部模仿之间存在显著负相关关系，表明更高的有意模仿与更低的自发模仿相关。
• 当参与者对所展示情绪的判断不确定性增加时，自发面部模仿也随之增加，表明模仿在不确定性中可能作为识别策略。
• 结果表明，在情绪识别任务中，面部模仿可能更多服务于认知功能（情绪推断）而非社交功能（亲密度建立），尤其当代理被感知为高度类人时。
• 尽管初始假设未获支持，本研究仍证实自发面部模仿仍是喜爱与亲密度的有效行为线索，且指令性模仿可作为其代理指标。
• 基于计算机视觉的AU强度检测方法在实地应用中表现可行，但其在光照和视角变化条件下的鲁棒性仍需进一步提升。