[Paper Review] Dynamics Learning with Cascaded Variational Inference for Multi-Step Manipulation

Kuan Fang, Yuke Zhu+3 more|arXiv (Cornell University)|Oct 29, 2019

Robot Manipulation and LearningEngineering27 references21 citations

TL;DR

This paper proposes a Factorized Dynamics Planner (FDP) that uses cascaded variational inference to learn hierarchical, disentangled representations for multi-step robotic manipulation. By separating high-level dynamics (planning) from low-level action sampling, the model improves long-horizon reasoning and achieves superior success rates on complex tasks like clearing, insertion, and crossing compared to baselines including Visual MPC, CVAE-SBMP, and SeCTAR.

ABSTRACT

The fundamental challenge of planning for multi-step manipulation is to find effective and plausible action sequences that lead to the task goal. We present Cascaded Variational Inference (CAVIN) Planner, a model-based method that hierarchically generates plans by sampling from latent spaces. To facilitate planning over long time horizons, our method learns latent representations that decouple the prediction of high-level effects from the generation of low-level motions through cascaded variational inference. This enables us to model dynamics at two different levels of temporal resolutions for hierarchical planning. We evaluate our approach in three multi-step robotic manipulation tasks in cluttered tabletop environments given high-dimensional observations. Empirical results demonstrate that the proposed method outperforms state-of-the-art model-based methods by strategically interacting with multiple objects.

Motivation & Objective

• To address the challenge of long-horizon robotic manipulation requiring complex, multi-object interactions and reasoning from visual observations.
• To improve sample efficiency and planning accuracy in multi-stage tasks by learning disentangled representations of dynamics and actions.
• To develop a hierarchical generative model that separates high-level dynamics from action sampling for better generalization and controllability.
• To evaluate the model on three complex tabletop manipulation tasks with dynamic constraints and sparse rewards.
• To ablate the impact of counterfactual regularization in improving robustness and zero-shot generalization.

Proposed method

• Uses a cascaded variational inference framework with two latent variables: $c$ for high-level dynamics (planning) and $z$ for low-level action sampling.
• Employs a conditional VAE to model the joint distribution $p(s_{t+1}, a_t | s_t, c)$, enabling disentangled generation of future states and actions.
• Introduces counterfactual regularization (CFR) to improve robustness by encouraging diverse, plausible trajectories during training.
• Trains the model end-to-end using a variational lower bound (ELBO) with amortized inference for scalable inference.
• Uses a shared encoder for visual observations to extract features for both $c$ and $z$, ensuring consistency across planning and action generation.
• Applies a self-consistent trajectory generation process where both states and actions are conditioned on the same latent code $c$, ensuring coherence in long-horizon plans.

Experimental results

Research questions

• RQ1 Can a disentangled, hierarchical variational inference model improve long-horizon planning in multi-step manipulation tasks compared to end-to-end baselines?
• RQ2 How does counterfactual regularization enhance the robustness and generalization of the planner across unseen task configurations?
• RQ3 To what extent does separating dynamics modeling from action sampling improve sample efficiency and success rates in complex, multi-object environments?
• RQ4 How does the FDP model compare to state-of-the-art methods like Visual MPC, CVAE-SBMP, and SeCTAR in terms of success rate and planning diversity?
• RQ5 Does the use of a 64-dimensional latent space (32 for $c$, 32 for $z$) provide sufficient capacity for effective planning while maintaining disentanglement?

Key findings

• The FDP model achieves higher success rates than Visual MPC, CVAE-SBMP, and SeCTAR across all three tasks: Clearing, Insertion, and Crossing.
• The ablation study shows that removing counterfactual regularization leads to a significant drop in performance, confirming its role in improving robustness and generalization.
• The model demonstrates superior zero-shot generalization to unseen task configurations, particularly in the Crossing task with randomized bridge layouts.
• The use of disentangled $c$ and $z$ latents enables more interpretable and controllable planning, as evidenced by consistent trajectory generation across diverse initial states.
• The model maintains high sample efficiency, generating effective long-horizon plans with fewer rollouts than baseline methods.
• The ablation with a 64-dimensional latent space (32+32) matches or exceeds performance of models using the same total dimensionality, validating the design choice of disentanglement.