[Paper Review] Towards Learning Efficient Maneuver Sets for Kinodynamic Motion Planning

Aravind Sivaramakrishnan, Zakary Littlefield+1 more|arXiv (Cornell University)|Jul 18, 2019

Robotic Path Planning Algorithms7 references4 citations

TL;DR

This paper proposes using learned maneuver sets—generated via neural networks—to enhance kinodynamic motion planners like DIRT, balancing exploitation (goal-directed) and exploration (diverse direction coverage). By training fully connected and convolutional networks to predict effective maneuvers from local obstacle and heuristic data, the method reduces solution time and cost, with the FC (All) model achieving the fastest convergence and best final solution quality in simulated environments.

ABSTRACT

Planning for systems with dynamics is challenging as often there is no local planner available and the only primitive to explore the state space is forward propagation of controls. In this context, tree sampling-based planners have been developed, some of which achieve asymptotic optimality by propagating random controls during each iteration. While desirable for the analysis, random controls result in slow convergence to high quality trajectories in practice. This short position statement first argues that if a kinodynamic planner has access to local maneuvers that appropriately balance an exploitation-exploration trade-off, the planner's per iteration performance is significantly improved. Generating such maneuvers during planning can be achieved by curating a large sample of random controls. This is, however, computationally very expensive. If such maneuvers can be generated fast, the planner's performance will also improve as a function of computation time. Towards objective, this short position statement argues for the integration of modern machine learning frameworks with state-of-the-art, informed and asymptotically optimal kinodynamic planners. The proposed approach involves using using neural networks to infer local maneuvers for a robotic system with dynamics, which properly balance the above exploitation-exploration trade-off. In particular, a neural network architecture is proposed, which is trained to reflect the choices of an online curation process, given local obstacle and heuristic information. The planner uses these maneuvers to efficiently explore the underlying state space, while still maintaining desirable properties. Preliminary indications in simulated environments and systems are promising but also point to certain challenges that motivate further research in this direction.

Motivation & Objective

• Improve the efficiency of asymptotically optimal kinodynamic planners by replacing random control sampling with learned, informed maneuvers.
• Address the slow convergence of random control propagation in tree-based planners by introducing a learned exploitation-exploration trade-off.
• Develop a machine learning pipeline that infers high-quality maneuvers from local state information (obstacles and heuristic guidance).
• Integrate neural networks into state-of-the-art planners (e.g., DIRT) to accelerate convergence while preserving asymptotic optimality.
• Evaluate the impact of learned maneuvers on both first-solution speed and final solution quality in diverse environments.

Proposed method

• Train a neural network to predict a set of 5 maneuvers (1 exploitative, 4 exploratory) from local state features: obstacle map and heuristic map.
• Use online curation of 1,000 random controls to generate a labeled dataset of optimal maneuvers for each state during DIRT planner execution.
• Employ two network architectures: fully connected (FC) and convolutional (Conv) to predict maneuver sets from local spatial features.
• Integrate the predicted maneuvers into the DIRT planner, prioritizing them during tree expansion to guide faster convergence.
• Use Euclidean distance to the goal as the heuristic function and trajectory duration as the cost function in evaluation.
• Train and evaluate on two environments: a 'Greedy' setting where heuristic guidance is effective and an 'Explore' setting where greedy behavior fails.

Experimental results

Research questions

• RQ1 Can learned maneuver sets significantly reduce the number of iterations required to find a solution in kinodynamic motion planning?
• RQ2 How does the performance of a planner using learned maneuvers compare to one using random controls in terms of solution cost and convergence speed?
• RQ3 Does the use of both exploitative and exploratory maneuvers in the learned set improve planner robustness in challenging environments?
• RQ4 How do different neural network architectures (FC vs. Conv) affect the efficiency and quality of the generated maneuvers?
• RQ5 What are the computational and collision-related trade-offs of using learned maneuvers versus random controls in real-time planning?

Key findings

• The DIRT-FC (All) model achieved the lowest number of iterations to find the first solution (1,876.67) and the best final solution cost (130.92) in the 'Explore' environment.
• In the 'Greedy' environment, DIRT-FC (All) found the first solution in 620 iterations and achieved the lowest final cost (45.47), outperforming random controls and single-mode predictions.
• Using only exploitative controls (e.g., DIRT-FC (Exploit)) led to failure in some runs in the 'Explore' environment, indicating insufficient exploration.
• The fully connected network achieved faster inference than random control sampling, though the convolutional network was slower without GPU acceleration.
• The learned maneuvers guided the robot effectively but resulted in more collisions than desired, indicating a need for improved training to prioritize collision-free paths.
• The current cost of neural network inference exceeds that of random control sampling, suggesting a need for hardware acceleration or model compression for real-time deployment.