[Paper Review] Learning Awareness Models

Brandon Amos, Laurent Dinh+8 more|arXiv (Cornell University)|Apr 17, 2018

Anomaly Detection Techniques and Applications36 references13 citations

TL;DR

This paper proposes learning dynamic body models from proprioceptive signals alone that implicitly represent external objects through their effects on the agent’s body. Despite no direct observation of the world, the model learns persistent, holistic object representations and enables accurate prediction of object properties and control policies in both simulation and real robotic hands.

ABSTRACT

We consider the setting of an agent with a fixed body interacting with an unknown and uncertain external world. We show that models trained to predict proprioceptive information about the agent's body come to represent objects in the external world. In spite of being trained with only internally available signals, these dynamic body models come to represent external objects through the necessity of predicting their effects on the agent's own body. That is, the model learns holistic persistent representations of objects in the world, even though the only training signals are body signals. Our dynamics model is able to successfully predict distributions over 132 sensor readings over 100 steps into the future and we demonstrate that even when the body is no longer in contact with an object, the latent variables of the dynamics model continue to represent its shape. We show that active data collection by maximizing the entropy of predictions about the body---touch sensors, proprioception and vestibular information---leads to learning of dynamic models that show superior performance when used for control. We also collect data from a real robotic hand and show that the same models can be used to answer questions about properties of objects in the real world. Videos with qualitative results of our models are available at https://goo.gl/mZuqAV.

Motivation & Objective

• To develop a framework where agents learn to represent external objects using only internal proprioceptive signals, without direct observation of the environment.
• To investigate whether predictive models of body states can implicitly capture persistent, holistic representations of external objects.
• To demonstrate that such models can be used for reasoning about object properties and for planning in control tasks.
• To validate the approach on both simulated and real robotic platforms, showing generalization to real-world dynamics.

Proposed method

• Train a deep predictive model on 132 proprioceptive sensor readings (joint angles, torques, contact forces, inertial measurements) from a robotic hand in simulation and reality.
• Use the model to predict future sensor states over 100 time steps, leveraging temporal consistency to infer object properties.
• Apply active data collection by maximizing prediction entropy to encourage exploration and improve model generalization.
• Extract latent representations from the dynamics model to infer object properties such as shape and orientation.
• Use the learned model for planning by optimizing trajectories in the latent space to achieve control objectives not seen during training.
• Transfer the same model architecture and training procedure to real robotic hand data, demonstrating real-world applicability.

Experimental results

Research questions

• RQ1 Can a model trained solely on proprioceptive signals learn persistent, holistic representations of external objects without direct observation of the world?
• RQ2 To what extent can predictive models of body dynamics infer object properties such as shape and orientation when the agent is not in contact with the object?
• RQ3 Does active data collection based on prediction uncertainty improve the quality of learned dynamics models for downstream reasoning and control?
• RQ4 Can the same model architecture and training procedure generalize from simulation to real-world robotic platforms?
• RQ5 Can the latent space of the dynamics model be used to answer diagnostic questions about external objects, such as object orientation?

Key findings

• The model successfully predicts 132 sensor readings over 100 steps into the future with high accuracy, even when the hand is not in contact with the object.
• Latent variables in the dynamics model continue to represent object shape and properties even after the hand loses contact, demonstrating persistent object awareness.
• Active data collection by maximizing prediction entropy led to models with superior performance in control and reasoning tasks.
• On the real robotic hand, the model achieved accurate prediction of object orientation (median error < 10°) using only proprioceptive data and model features.
• The model enabled planning in the latent space to achieve objectives not seen during training, demonstrating reasoning beyond direct observation.
• Bootstrap analysis showed that model features significantly outperformed raw sensor data in predicting object orientation, with 95% confidence intervals indicating statistical improvement.