[Paper Review] Learning Particle Dynamics for Manipulating Rigid Bodies, Deformable Objects, and Fluids

Yunzhu Li, Jiajun Wu+3 more|arXiv (Cornell University)|Oct 3, 2018

Robot Manipulation and Learning35 references105 citations

TL;DR

This paper learns a differentiable particle-based dynamics model (DPI-Nets) with dynamic graphs and hierarchical propagation to simulate and control rigid, deformable, and fluid materials, enabling real-world manipulation and online adaptation.

ABSTRACT

Real-life control tasks involve matters of various substances---rigid or soft bodies, liquid, gas---each with distinct physical behaviors. This poses challenges to traditional rigid-body physics engines. Particle-based simulators have been developed to model the dynamics of these complex scenes; however, relying on approximation techniques, their simulation often deviates from real-world physics, especially in the long term. In this paper, we propose to learn a particle-based simulator for complex control tasks. Combining learning with particle-based systems brings in two major benefits: first, the learned simulator, just like other particle-based systems, acts widely on objects of different materials; second, the particle-based representation poses strong inductive bias for learning: particles of the same type have the same dynamics within. This enables the model to quickly adapt to new environments of unknown dynamics within a few observations. We demonstrate robots achieving complex manipulation tasks using the learned simulator, such as manipulating fluids and deformable foam, with experiments both in simulation and in the real world. Our study helps lay the foundation for robot learning of dynamic scenes with particle-based representations.

Motivation & Objective

• Address the challenge of modeling diverse object dynamics (rigid, deformable, fluids) for robust robotic manipulation.
• Propose a differentiable, particle-based simulator that generalizes across materials.
• Introduce dynamic interaction graphs and hierarchical propagation to capture short- and long-range interactions.
• Enable online adaptation and model-based control (MPC/shooting) using the learned dynamics.

Proposed method

• Extend interaction networks to particle-level dynamics with dynamic graph construction.
• Introduce hierarchical clustering of particles to model long-range dependencies for certain objects.
• Use multi-stage propagation and dynamic graphs to capture instantaneous and long-range interactions.
• Model deformable and fluid materials with state-specific motion predictors and resting positions where applicable.
• Apply model-based control via trajectory optimization (shooting method) and MPC with online system identification.

Experimental results

Research questions

• RQ1 Can a differentiable, particle-based dynamics model generalize across rigid, deformable, and fluid objects?
• RQ2 Do dynamic, on-the-fly constructed interaction graphs improve forward prediction over static graphs or complete graphs?
• RQ3 Can the learned DPI-Nets enable effective control and online adaptation in both simulation and real-world tasks?
• RQ4 How does hierarchical propagation affect learning and control for objects with long-range dependencies?
• RQ5 What is the impact of online system identification on task performance under unknown physical parameters?

Key findings

• DPI-Nets achieve lower forward-simulation MSE than baselines (FluidFall: 0.15±0.03 vs HRN 0.21±0.04; BoxBath: 2.03±0.41 vs HRN 3.62±0.40).
• In FluidShake and RiceGrip, DPI-Net outperforms HRN and DPI-Net without hierarchy, with values: FluidShake 1.89±0.36 (both w/ and w/o hierarchy) and RiceGrip 0.13±0.07 (DPI-Net) versus 0.29±0.13 (w/o hierarchy).
• Dynamic graphs outperform static graphs (IN) and enable modeling of fluids and multi-state interactions (BoxBath, FluidShake).
• Online adaptation via SGD-based online system identification improves manipulation in real world (RiceGrip in Real World).
• The approach enables control tasks such as molding deformable objects and guiding fluids to target configurations using MPC and gradient-based trajectory optimization.