[Paper Review] Object-centric Forward Modeling for Model Predictive Control

Yufei Ye, Dhiraj Gandhi+2 more|arXiv (Cornell University)|Oct 8, 2019

Reinforcement Learning in Robotics33 references6 citations

TL;DR

This paper proposes an object-centric forward model that represents scenes as spatially localized objects with implicit visual features, enabling accurate prediction of action effects and interactions. By leveraging this structured representation in model predictive control with a correction module, the method achieves sample-efficient, robust planning for distant goals in both simulation and real-world robotic manipulation tasks, outperforming pixel-space and implicit baseline models.

ABSTRACT

We present an approach to learn an object-centric forward model, and show that this allows us to plan for sequences of actions to achieve distant desired goals. We propose to model a scene as a collection of objects, each with an explicit spatial location and implicit visual feature, and learn to model the effects of actions using random interaction data. Our model allows capturing the robot-object and object-object interactions, and leads to more sample-efficient and accurate predictions. We show that this learned model can be leveraged to search for action sequences that lead to desired goal configurations, and that in conjunction with a learned correction module, this allows for robust closed loop execution. We present experiments both in simulation and the real world, and show that our approach improves over alternate implicit or pixel-space forward models. Please see our project page (https://judyye.github.io/ocmpc/) for result videos.

Motivation & Objective

• To develop a forward model that captures object interactions and action effects in structured, object-centric representations for long-horizon robotic planning.
• To improve sample efficiency and prediction accuracy over pixel-space or implicit representation-based forward models.
• To enable robust closed-loop control by integrating a learned correction module that refines predictions using real-time observations.
• To demonstrate generalization of the approach in both simulated and real-world robotic manipulation environments.
• To reduce reliance on perfect dynamics models by learning interaction-aware, structured world representations from random interaction data.

Proposed method

• Represent scenes as collections of objects with explicit spatial locations and implicit visual features, enabling structured modeling of object dynamics.
• Train a forward model to predict changes in object states (position and feature) under actions, using a graph neural network to model interactions between robot and objects, and among objects.
• Use model predictive control (MPC) with cross-entropy method (CEM) to search for action sequences that reach a desired goal configuration.
• Integrate a correction module that re-estimates object states using intermediate observations, enabling closed-loop execution and error correction.
• Leverage random interaction data for self-supervised training, avoiding the need for precise ground-truth dynamics.
• Unroll predictions over T steps using the forward model to simulate long-horizon action sequences.

Experimental results

Research questions

• RQ1 Can an object-centric forward model improve long-horizon planning accuracy compared to pixel-space or implicit representation models?
• RQ2 How does modeling object interactions enhance prediction accuracy and sample efficiency in robotic manipulation tasks?
• RQ3 To what extent does a learned correction module improve robustness in closed-loop execution under model inaccuracies?
• RQ4 Can the method generalize to real-world environments with complex object interactions and large motion per action?
• RQ5 How does the structured inductive bias of object-centric representation support better generalization than end-to-end pixel-based modeling?

Key findings

• The proposed object-centric forward model outperforms both pixel-space and implicit representation baselines in long-horizon planning, especially in scenarios requiring object interaction and collision awareness.
• Ablation studies show that removing the interaction network leads to faster initial progress but worse long-term performance, confirming the importance of modeling interactions.
• The correction module significantly reduces prediction error accumulation over time, with error decreasing by up to 50% in 10-step unrollings when intermediate observations are used.
• Visualizations show that the model learns to plan non-greedy, collision-avoiding trajectories, such as pushing one object around another, which simpler models fail to capture.
• In real-world experiments, the flow-based baseline fails to predict large motions, while the proposed model accurately predicts object movements due to interaction modeling.
• The CEM-based planner converges to optimal trajectories over iterations, with elite action sequences clearly avoiding obstacles and achieving the goal.