[Paper Review] Morphology-Independent Facial Expression Imitation for Human-Face Robots

Xu Chen, Rui Gao+5 more|arXiv (Cornell University)|Mar 7, 2026

Social Robot Interaction and HRI0 citations

TL;DR

Proposes a self-supervised, morphology-independent framework to decouple facial expressions from morphology for faithful expression imitation on a custom human-face robot, Pengrui. Demonstrates superior representation disentanglement and actuator-command mapping over baselines.

ABSTRACT

Accurate facial expression imitation on human-face robots is crucial for achieving natural human-robot interaction. Most existing methods have achieved photorealistic expression imitation through mapping 2D facial landmarks to a robot's actuator commands. Their imitation of landmark trajectories is susceptible to interference from facial morphology, which would lead to a performance drop. In this paper, we propose a morphology-independent expression imitation method that decouples expressions from facial morphology to eliminate morphological influence and produce more realistic expressions for human-face robots. Specifically, we construct an expression decoupling module to learn expression semantics by disentangling the expression representation from the morphology representation in a self-supervised manner. We devise an expression transfer module to map the representations to the robot's actuator commands through a learning objective of perceiving expression errors, producing accurate facial expressions based on the learned expression semantics. To support experimental validation, a custom-designed and highly expressive human-face robot, namely Pengrui, is developed to serve as an experimental platform for realistic expression imitation. Extensive experiments demonstrate that our method enables the human-face robot to reproduce a wide range of human-like expressions effectively. All code and implementation details of the robot will be released.

Motivation & Objective

• Motivation: facial expression imitation on human-face robots is hindered by morphology-driven interference in expression representations.
• Goal: decouple expression from morphology to enable robust, realistic imitation across diverse morphologies.
• Approach: learn expression semantics via self-supervised disentanglement and map them to robot actuators with perceptual-error driven transfer.
• Contribution: a new morphology-independent framework and a dedicated high-fidelity robot (Pengrui) for real-world validation.

Proposed method

• Expression Decoupling Module (EDM) that extracts disentangled expression e, pose p, and morphology m from images using an encoder–decoder with FLAME-based self-supervision.
• Expression Transfer Module (ETM) that maps e to 22 robot actuator commands via a learned encoder, and uses a frozen decoder (ETM-INV) to optimize perceptual fidelity between intended and realized expressions.
• Self-supervision signals come from aligning 2D landmarks derived from FLAME-generated faces with real input landmarks to train EDM without annotated data.
• Training leverages synthetic FLAME data for parameter-to-face reconstruction and real robot data to train ETM with an expression reconstruction loss to ensure perceptual alignment.
• Robot Pengrui: a 32-actuator, high-DoF human-face robot with a rigid-linkage actuation system and silicone skin, used for validating real-world imitation.

Experimental results

Research questions

• RQ1 Can expressions be effectively decoupled from facial morphology to reduce morphology-induced interference in expression imitation?
• RQ2 Does a morphology-independent representation improve cross-morphology expression fidelity on humanoid-face robots?
• RQ3 How well can the learned expression representations be mapped to robot actuator commands to reproduce natural expressions across diverse morphologies?
• RQ4 What is the impact of the morphology-independent approach on real-world robot臨imitation performance compared to landmark-based baselines?

Key findings

• The morphology-independent representations show substantially lower variability across morphologies for the same expression (CV reduction vs landmark-based baselines).
• The EDM (ours) significantly improves overall, expression, and morphology representation accuracy over RG baselines (lower MSE/MAE by large margins in Table II).
• The full EDM+ETM pipeline achieves the best actuator-command prediction accuracy compared with several baselines, including random and nearest-neighbor variants (Table III).
• Real-robot experiments with Pengrui demonstrate faithful reproduction of a wide range of human expressions across individuals with different morphologies.