[Paper Review] Deep Model Predictive Control with Stability Guarantees

Prabhat K. Mishra, Mateus V. Gasparino+2 more|arXiv (Cornell University)|Apr 15, 2021

Advanced Control Systems Optimization46 references4 citations

TL;DR

This paper proposes a deep learning-enhanced model predictive control (deep-MPC) framework for nonlinear, control-affine discrete-time systems with unstructured, bounded uncertainties. By integrating a dual-timescale neural network adaptation with a tube-based MPC, the method ensures input-to-state stability, recursive feasibility, and convergence to the origin under verifiable conditions, while maintaining constraint satisfaction during learning transients.

ABSTRACT

This paper presents a deep learning based model predictive control algorithm for control affine nonlinear discrete time systems with matched and bounded state dependent uncertainties of unknown structure. Since the structure of uncertainties is not known, a deep learning based adaptive mechanism is utilized to mitigate disturbances. In order to avoid any unwanted behavior during the learning phase, a tube based model predictive controller is employed, which ensures satisfaction of constraints and input-to-state stability of the closed-loop states. In addition, the proposed approach guarantees the convergence of states to origin under certain verifiable conditions. To ensure stability and undesirable learning transients, a dual-timescale adaptation mechanism is proposed, where the weights of the last layer of the neural network are updated each time instant while the inner layers are trained on a slower timescale using training data collected online and selectively stored in a buffer on the basis of singular value maximization criterion. Our results are validated through numerical experiments on wing-rock dynamics. These results indicate that the proposed deep-MPC architecture is effective in learning to control safety critical systems without suffering instability drawbacks.

Motivation & Objective

• Address the challenge of ensuring stability and constraint satisfaction during learning transients in deep learning-based control for safety-critical systems.
• Overcome limitations of purely data-driven controllers that may lead to instability or constraint violations during adaptation.
• Integrate deep neural networks (DNNs) with model predictive control (MPC) to leverage function approximation and real-time optimization while preserving safety.
• Develop a dual-timescale adaptation mechanism to decouple fast output-layer learning from slower hidden-layer training, enabling real-time implementability.
• Guarantee asymptotic stability and recursive feasibility using Lyapunov-based analysis and a tube-based MPC formulation with disturbance rejection capability.

Proposed method

• Employ a deep neural network (DNN) with fixed hidden layers and adaptive output layer weights to approximate unknown, unstructured system uncertainties.
• Implement a dual-timescale adaptation: output-layer weights are updated at every time step using a discrete adaptive law, while hidden-layer weights are updated slowly using online data stored in a buffer.
• Use a singular value maximization criterion to selectively store training data in the buffer, improving sample efficiency and learning quality.
• Integrate the DNN's disturbance estimate into a tube-based MPC framework that ensures recursive feasibility and input-to-state stability (ISS) of the closed-loop system.
• Formulate a Lyapunov function combining the MPC cost and the DNN weight error to prove convergence and stability, with a composite Lyapunov function for the full system.
• Enforce constraints via a robust tube around the nominal trajectory, where the tube size is bounded by the DNN's approximation error and system uncertainty bounds.

Experimental results

Research questions

• RQ1 Can a deep learning-based controller be integrated with MPC to ensure stability and constraint satisfaction during online learning for nonlinear systems with unstructured uncertainties?
• RQ2 How can the learning process be decoupled from real-time control to avoid instability and ensure implementability in safety-critical systems?
• RQ3 What conditions guarantee the convergence of the closed-loop states to the origin while maintaining recursive feasibility and input-to-state stability?
• RQ4 How can online training data be efficiently selected and stored to improve DNN generalization without compromising safety?
• RQ5 Can a provably stable and safe control architecture be designed for systems with matched, bounded, state-dependent uncertainties using deep learning and MPC in tandem?

Key findings

• The proposed deep-MPC framework ensures input-to-state stability (ISS) of the closed-loop system under bounded, unstructured uncertainties.
• Recursive feasibility of the MPC optimization is guaranteed via a tube-based formulation that accounts for the DNN's disturbance estimation error.
• The system states converge to the origin asymptotically, as proven by a composite Lyapunov function showing exponential decay of the state norm.
• The dual-timescale adaptation mechanism enables real-time implementation by updating output-layer weights at each time step while training hidden-layer weights slowly using a curated buffer of online data.
• Numerical experiments on wing-rock dynamics demonstrate effective learning of complex nonlinear dynamics without instability or constraint violations.
• Theoretical analysis confirms that the sum of the fourth power of the state norm over time is bounded, implying asymptotic convergence to the origin under the derived stability conditions.