[Paper Review] Object-Oriented Dynamics Predictor

Guangxiang Zhu, Zhiao Huang+1 more|arXiv (Cornell University)|May 25, 2018

Reinforcement Learning in Robotics18 citations

TL;DR

This paper proposes Object-Oriented Dynamics Predictor (OODP), an end-to-end, unsupervised neural network that decomposes environments into objects and predicts action-conditioned dynamics using class-specific, CNN-based object relations. OODP achieves strong generalization across novel object layouts and appearances, outperforming prior methods in zero-shot generalization and learning semantically interpretable dynamics models.

ABSTRACT

Generalization has been one of the major challenges for learning dynamics models in model-based reinforcement learning. However, previous work on action-conditioned dynamics prediction focuses on learning the pixel-level motion and thus does not generalize well to novel environments with different object layouts. In this paper, we present a novel object-oriented framework, called object-oriented dynamics predictor (OODP), which decomposes the environment into objects and predicts the dynamics of objects conditioned on both actions and object-to-object relations. It is an end-to-end neural network and can be trained in an unsupervised manner. To enable the generalization ability of dynamics learning, we design a novel CNN-based relation mechanism that is class-specific (rather than object-specific) and exploits the locality principle. Empirical results show that OODP significantly outperforms previous methods in terms of generalization over novel environments with various object layouts. OODP is able to learn from very few environments and accurately predict dynamics in a large number of unseen environments. In addition, OODP learns semantically and visually interpretable dynamics models.

Motivation & Objective

• To address the poor generalization of pixel-level dynamics models in novel environments with different object layouts.
• To enable end-to-end, unsupervised learning of object-level dynamics conditioned on actions and object-to-object relations.
• To design a relation mechanism that is class-specific and exploits locality for improved generalization and interpretability.
• To learn semantically and visually interpretable dynamics models that generalize across unseen environments.
• To demonstrate robustness to object appearance variations and natural image inputs.

Proposed method

• OODP uses a self-supervised, end-to-end neural network to decompose visual observations into objects via an object detector.
• It employs a novel CNN-based relation mechanism that formulates class-specific object masks instead of object-specific vectors, enabling generalization across object instances.
• The relation mechanism exploits the locality principle through neighborhood cropping and CNNs to model spatial interactions between objects.
• Object-level dynamics are predicted by conditioning on both actions and learned object-to-object relations, using a spatial transformer network (STN) for spatial transformation.
• The model is trained in an unsupervised manner using reconstruction loss on future frames, without requiring explicit object annotations.
• The framework integrates object detection, relation modeling, and dynamics prediction in a unified architecture, enabling joint learning of perception and dynamics.

Experimental results

Research questions

• RQ1 Can an end-to-end, unsupervised neural network learn dynamics models that generalize across novel object layouts?
• RQ2 How does a class-specific, locality-aware relation mechanism improve generalization in dynamics prediction?
• RQ3 Can object-oriented dynamics learning lead to semantically and visually interpretable models?
• RQ4 To what extent can the model generalize to environments with different object appearances and layouts?
• RQ5 Can the model handle real-world natural image inputs, such as Mars rover navigation scenarios?

Key findings

• OODP achieves 94% accuracy and 0.28 RMSE in 5-to-10 generalization over novel object layouts (S0-S6), significantly outperforming prior methods.
• In the Mars rover navigation domain, OODP achieves 92% accuracy (n-error) in unseen environments, compared to 75% for CDNA and 12% for the AC model.
• OODP maintains high performance (accuracy > 0.88) even when object appearances differ from training data, demonstrating robustness to appearance variations.
• Visualization of learned masks shows that OODP successfully identifies key objects and their relations in unseen environments, enabling reuse of object-level knowledge.
• The model learns interpretable dynamics by decomposing scenes into meaningful objects and relations, with spatial attention focused on relevant moving and static objects.
• OODP generalizes effectively from very few training environments, accurately predicting dynamics in a large number of unseen environments.