[Paper Review] Stein Variational Model Predictive Control

Alexander Lambert, Adam Fishman+3 more|arXiv (Cornell University)|Nov 15, 2020

Advanced Control Systems OptimizationEngineering37 references18 citations

TL;DR

This paper proposes Stein Variational Model Predictive Control (SV-MPC), a novel MPC framework that represents control solutions as nonparametric posterior distributions over control parameters using Stein variational gradient descent (SVGD). By formulating MPC as a Bayesian inference problem, SV-MPC effectively handles complex, multi-modal, and non-convex optimal control problems, demonstrating superior performance in challenging robotic tasks such as navigation and manipulation with improved success rates and lower cost compared to MPPI and CEM.

ABSTRACT

Decision making under uncertainty is critical to real-world, autonomous systems. Model Predictive Control (MPC) methods have demonstrated favorable performance in practice, but remain limited when dealing with complex probability distributions. In this paper, we propose a generalization of MPC that represents a multitude of solutions as posterior distributions. By casting MPC as a Bayesian inference problem, we employ variational methods for posterior computation, naturally encoding the complexity and multi-modality of the decision making problem. We present a Stein variational gradient descent method to estimate the posterior directly over control parameters, given a cost function and observed state trajectories. We show that this framework leads to successful planning in challenging, non-convex optimal control problems.

Motivation & Objective

• To address the limitations of traditional MPC in handling complex, non-Gaussian, multi-modal distributions over control inputs.
• To enable effective planning in non-convex optimal control problems with obstacles, multiple goals, or non-convex constraints.
• To develop a scalable, online MPC method that adapts to non-stationary posterior distributions in real-time robotic decision-making.
• To leverage variational inference and SVGD for efficient, nonparametric approximation of the posterior over control parameters.
• To provide a unified framework that generalizes to trajectory optimization and integrates smoothly with existing MPC and SOC methods.

Proposed method

• Formulates MPC as a Bayesian inference problem, where the posterior is defined over control parameters rather than joint state-action distributions.
• Uses relative-entropy minimization to approximate the true posterior, enabling variational inference over control sequences.
• Employs Stein Variational Gradient Descent (SVGD) to iteratively update a set of particles representing the posterior distribution over control inputs.
• Defines a prior over control sequences using a Gaussian process-like structure with Markovian dependencies via a lower-triangular matrix transformation.
• Integrates a cost function that includes obstacle avoidance (via a bi-modal Gaussian mixture map) and goal-reaching penalties.
• Applies the method in an online, receding-horizon fashion, with warm-starting and iterative optimization per time step.

Experimental results

Research questions

• RQ1 Can a nonparametric variational inference method like SVGD effectively represent multi-modal control policies in complex, non-convex optimal control problems?
• RQ2 How does SV-MPC compare to sampling-based MPC methods like MPPI and CEM in terms of success rate and cost minimization under uncertainty?
• RQ3 Can the proposed framework handle non-stationary, rapidly changing posterior distributions common in real-time robotic control?
• RQ4 What is the impact of particle count and optimization iterations on the performance and convergence of the SV-MPC controller?
• RQ5 How does the choice of prior over control sequences affect trajectory smoothness and planning performance?

Key findings

• In the planar navigation task with a 4x4 obstacle grid, SV-MPC achieved a 96% success rate with an average cost of 20.7×10³ using 32 particles, outperforming MPPI and CEM, which had 64% success rates and higher costs.
• With 12 particles, SV-MPC maintained 96% success rate and only slightly increased cost to 21.8×10³, demonstrating robustness to particle count reduction.
• The 6-particle SV-MPC version showed a drop in success rate to 84%, indicating that performance is sensitive to particle count but remains competitive with MPPI and CEM at higher counts.
• SV-MPC with 32 particles achieved lower average cost (20.7×10³) than MPPI (26.5×10³) and CEM (25.4×10³), indicating better cost minimization in the presence of obstacles.
• The method successfully handled non-convexity in the Half-cheetah and manipulation tasks, showing scalability to higher-dimensional control problems.
• The use of a smooth, Markovian prior over control sequences led to smoother state trajectories, analogous to GP-priors in previous work.