[Paper Review] Learning Plannable Representations with Causal InfoGAN

Thanard Kurutach, Aviv Tamar+3 more|arXiv (Cornell University)|Jul 24, 2018

Multimodal Machine Learning ApplicationsComputer Science33 references91 citations

TL;DR

Causal InfoGAN learns low-dimensional, plannable representations from high-dimensional observations and enables goal-directed visual planning by mapping observations to abstract states and planning in that latent space. It then decodes a plan back into a sequence of observations.

ABSTRACT

In recent years, deep generative models have been shown to 'imagine' convincing high-dimensional observations such as images, audio, and even video, learning directly from raw data. In this work, we ask how to imagine goal-directed visual plans -- a plausible sequence of observations that transition a dynamical system from its current configuration to a desired goal state, which can later be used as a reference trajectory for control. We focus on systems with high-dimensional observations, such as images, and propose an approach that naturally combines representation learning and planning. Our framework learns a generative model of sequential observations, where the generative process is induced by a transition in a low-dimensional planning model, and an additional noise. By maximizing the mutual information between the generated observations and the transition in the planning model, we obtain a low-dimensional representation that best explains the causal nature of the data. We structure the planning model to be compatible with efficient planning algorithms, and we propose several such models based on either discrete or continuous states. Finally, to generate a visual plan, we project the current and goal observations onto their respective states in the planning model, plan a trajectory, and then use the generative model to transform the trajectory to a sequence of observations. We demonstrate our method on imagining plausible visual plans of rope manipulation.

Motivation & Objective

• Motivate and address how to imagine goal-directed visual plans from high-dimensional observations.
• Learn a low-dimensional, planning-friendly representation that captures the causal structure of the data.
• Integrate representation learning with planning to produce a sequence of observations that transitions from start to goal.

Proposed method

• Train a causal InfoGAN that factors observations into a structured latent planning system and a noise component.
• Use a mutual-information objective to ensure the abstract states s, s' capture causal transitions that explain the data.
• Support both discrete (one-hot or binary) and continuous latent planning systems with compatible planning algorithms.
• Encode observations to latent states via Q(s|o) or through latent-space optimization to handle high-dimensional observations.
• Decode a latent state trajectory into a sequence of observations using a conditional GAN generator, selecting best trajectories with a discriminator or novelty detector.
• Optimize with a variational lower bound I_VLB to approximate mutual information between latent transitions and generated observations.

Experimental results

Research questions

• RQ1 How can we learn representations that are both expressive for high-dimensional data and structured for efficient planning?
• RQ2 Can a GAN-based model with a planning-focused latent space generate plausible walkthroughs from a start to a goal observation?
• RQ3 How can latent planning systems be designed (discrete or continuous) to be compatible with standard planning algorithms?
• RQ4 What strategies improve encoding of real observations to latent states in high-dimensional domains?
• RQ5 How effective are the learned representations and generated walkthroughs for tasks like rope manipulation?

Key findings

• Causal InfoGAN can learn abstract states that align with causal transitions and support planning in the latent space.
• The framework supports discrete and continuous latent planning systems that are compatible with planning algorithms like Dijkstra and linear interpolation.
• The method can generate plausible visual walkthroughs from start to goal, demonstrated on rope manipulation scenarios using real image data.
• Encoding strategies for high-dimensional observations (search-based latent mapping or learned Q) improve state mapping beyond simple discriminators trained on generated data.
• A variational lower bound facilitates training the mutual information objective, enabling end-to-end optimization of the model.