[Paper Review] Gated Path Planning Networks

Lisa Lee, Emilio Parisotto+3 more|arXiv (Cornell University)|Jun 17, 2018

Robotic Path Planning Algorithms14 references6 citations

TL;DR

This paper proposes Gated Path Planning Networks (GPPNs), a differentiable path planning module that replaces the unconventional recurrent update in Value Iteration Networks (VINs) with a standard LSTM-based gated recurrent unit. By reframing VIN as a convolutional-recurrent network, GPPNs achieve faster learning, improved training stability, reduced hyperparameter and random seed sensitivity, and better generalization—outperforming VIN across 2D mazes and a 3D ViZDoom environment using only RGB observations.

ABSTRACT

Value Iteration Networks (VINs) are effective differentiable path planning modules that can be used by agents to perform navigation while still maintaining end-to-end differentiability of the entire architecture. Despite their effectiveness, they suffer from several disadvantages including training instability, random seed sensitivity, and other optimization problems. In this work, we reframe VINs as recurrent-convolutional networks which demonstrates that VINs couple recurrent convolutions with an unconventional max-pooling activation. From this perspective, we argue that standard gated recurrent update equations could potentially alleviate the optimization issues plaguing VIN. The resulting architecture, which we call the Gated Path Planning Network, is shown to empirically outperform VIN on a variety of metrics such as learning speed, hyperparameter sensitivity, iteration count, and even generalization. Furthermore, we show that this performance gap is consistent across different maze transition types, maze sizes and even show success on a challenging 3D environment, where the planner is only provided with first-person RGB images.

Motivation & Objective

• To address training instability, random seed sensitivity, and hyperparameter sensitivity in Value Iteration Networks (VINs).
• To investigate whether the inductive biases of VINs are necessary for effective differentiable path planning.
• To improve optimization and generalization in end-to-end differentiable navigation architectures.
• To evaluate performance on diverse environments, including 2D mazes and a 3D video game environment with RGB-only observations.

Proposed method

• Reframe VIN as a convolutional-recurrent network to enable the use of standard gated recurrent units like LSTM.
• Replace the VIN's unconventional update (convolution + max-pooling) with a gated recurrent update mechanism.
• Use an LSTM-based update to maintain and update state representations across iterations, improving gradient flow.
• Train the GPPN end-to-end using supervised imitation of optimal paths, with shared weights across spatial positions.
• Apply the GPPN to both 2D grid-world mazes and a 3D ViZDoom environment with first-person RGB inputs.
• Use kernel size and iteration count as controllable hyperparameters to assess scalability and efficiency.

Experimental results

Research questions

• RQ1 Can replacing the VIN's unconventional recurrent update with a standard gated recurrent unit improve training stability and convergence?
• RQ2 Does the GPPN reduce sensitivity to random seeds and hyperparameter choices compared to VIN?
• RQ3 Can the GPPN generalize better than VIN with less training data?
• RQ4 Does the GPPN maintain or improve performance on larger kernel sizes and fewer iterations compared to VIN?
• RQ5 Can the GPPN successfully plan in complex 3D environments using only RGB observations?

Key findings

• GPPNs achieve significantly faster learning, reaching high performance in just a few training epochs, whereas VIN requires longer training.
• GPPNs show reduced training instability, with fewer oscillations between high and low performance across epochs.
• GPPNs exhibit lower sensitivity to random seeds and hyperparameters, leading to more consistent performance across runs.
• GPPNs generalize better with less training data, and the performance gap over VIN increases as data size decreases.
• GPPNs outperform VIN on 2D mazes across various maze sizes, transition types, and kernel sizes, including larger kernels that destabilize VIN.
• In the 3D ViZDoom environment, GPPNs successfully plan using only first-person RGB images, demonstrating robustness to partial observability and complex visual input.