[Paper Review] Robot self/other distinction: active inference meets neural networks learning in a mirror

Pablo Lanillos, Jordi Pagès+1 more|arXiv (Cornell University)|Apr 11, 2020

Action Observation and Synchronization34 references19 citations

TL;DR

This paper proposes a neural network-based algorithm that enables a robot to distinguish its own actions from those of others using active inference and free energy minimization. By learning sensorimotor contingencies between self-generated actions and visual/body state changes, the robot accumulates evidence for self-recognition in a mirror without relying on appearance cues, achieving robust self/other distinction across diverse conditions.

ABSTRACT

Self/other distinction and self-recognition are important skills for interacting with the world, as it allows humans to differentiate own actions from others and be self-aware. However, only a selected group of animals, mainly high order mammals such as humans, has passed the mirror test, a behavioural experiment proposed to assess self-recognition abilities. In this paper, we describe self-recognition as a process that is built on top of body perception unconscious mechanisms. We present an algorithm that enables a robot to perform non-appearance self-recognition on a mirror and distinguish its simple actions from other entities, by answering the following question: am I generating these sensations? The algorithm combines active inference, a theoretical model of perception and action in the brain, with neural network learning. The robot learns the relation between its actions and its body with the effect produced in the visual field and its body sensors. The prediction error generated between the models and the real observations during the interaction is used to infer the body configuration through free energy minimization and to accumulate evidence for recognizing its body. Experimental results on a humanoid robot show the reliability of the algorithm for different initial conditions, such as mirror recognition in any perspective, robot-robot distinction and human-robot differentiation.

Motivation & Objective

• To enable robots to perform non-appearance self-recognition in a mirror using sensorimotor learning and active inference.
• To address the challenge of self/other distinction in dynamic, real-world scenarios where visual and proprioceptive cues are partial or ambiguous.
• To ground self-recognition in unconscious body perception mechanisms rather than high-level cognitive awareness.
• To develop a scalable, biologically plausible framework that avoids reliance on predefined body segmentation or appearance models.
• To demonstrate robustness across varying mirror perspectives, robot-robot, and human-robot interaction scenarios.

Proposed method

• The robot learns forward models mapping its motor commands to expected visual and proprioceptive outcomes using neural networks.
• Prediction errors between predicted and actual sensory states are computed and minimized via free energy optimization, driving inference of body configuration.
• A probabilistic model computes the marginal likelihood of sensory data under the hypothesis that the robot generated the action, accumulating evidence for self-recognition.
• The double comparator model integrates prediction error (first comparator) and spatiotemporal contingency (second comparator) to improve self/other distinction.
• Sensorimotor uncertainty is explicitly modeled, allowing robust inference even under partial or noisy observations.
• The framework uses variational inference to approximate Bayesian model evidence, enabling online learning and real-time self-recognition.

Experimental results

Research questions

• RQ1 Can a robot distinguish its own actions from those of others in a mirror using only non-appearance sensory cues?
• RQ2 How can active inference and free energy minimization be used to learn and update sensorimotor models for self-recognition?
• RQ3 To what extent can self/other distinction emerge from sensorimotor contingency learning without relying on visual appearance or pre-segmented body parts?
• RQ4 Can the proposed model maintain robustness across different mirror perspectives and dynamic interactions with other agents?
• RQ5 How does the integration of prediction error and spatiotemporal contingency improve the reliability of self-recognition compared to single-mechanism approaches?

Key findings

• The robot successfully distinguished its own actions from those of a human or another robot in mirror experiments, even when visual appearances were similar.
• Low prediction error during self-generated movements led to high confidence in self-recognition, while higher error reduced the probability of self-identification.
• The algorithm demonstrated robustness across different initial mirror perspectives and dynamic interaction conditions.
• The system accumulated temporal evidence through marginal likelihood computation, enabling reliable self-recognition without requiring explicit appearance matching.
• An unexpected side effect was the emergence of mirroring-like behavior when the robot’s hand was observed in the mirror, suggesting imitative tendencies through inference.
• The method achieved self/other distinction without requiring complex visual segmentation or prior knowledge of body shape, relying only on action-effect contingencies.