[Paper Review] Deep Model Predictive Control with Online Learning for Complex Physical Systems

Katharina Bieker, Sebastian Peitz+3 more|arXiv (Cornell University)|May 24, 2019

Advanced Control Systems Optimization28 references34 citations

TL;DR

The authors present DeepMPC, a sensor-based, data-driven MPC framework that uses a deep RNN to predict control-relevant quantities and integrates it into MPC with online updates to achieve real-time control of complex fluid flows.

ABSTRACT

The control of complex systems is of critical importance in many branches of science, engineering, and industry. Controlling an unsteady fluid flow is particularly important, as flow control is a key enabler for technologies in energy (e.g., wind, tidal, and combustion), transportation (e.g., planes, trains, and automobiles), security (e.g., tracking airborne contamination), and health (e.g., artificial hearts and artificial respiration). However, the high-dimensional, nonlinear, and multi-scale dynamics make real-time feedback control infeasible. Fortunately, these high-dimensional systems exhibit dominant, low-dimensional patterns of activity that can be exploited for effective control in the sense that knowledge of the entire state of a system is not required. Advances in machine learning have the potential to revolutionize flow control given its ability to extract principled, low-rank feature spaces characterizing such complex systems. We present a novel deep learning model predictive control (DeepMPC) framework that exploits low-rank features of the flow in order to achieve considerable improvements to control performance. Instead of predicting the entire fluid state, we use a recurrent neural network (RNN) to accurately predict the control relevant quantities of the system. The RNN is then embedded into a MPC framework to construct a feedback loop, and incoming sensor data is used to perform online updates to improve prediction accuracy. The results are validated using varying fluid flow examples of increasing complexity.

Motivation & Objective

• Leverage low-dimensional, dominant patterns in high-dimensional fluid flows for effective control without full state knowledge.
• Develop a DeepMPC architecture that predicts control-relevant quantities rather than the full state.
• Enable online updates to the surrogate model using incoming sensor data to improve prediction accuracy in real time.
• Demonstrate performance on increasingly complex flow-control scenarios and assess robustness to chaotic dynamics.

Proposed method

• Formulate MPC over a prediction horizon N to minimize tracking error and control effort using a surrogate dynamics model for observable z = f(y).
• Approximate the surrogate dynamics with a deep recurrent neural network (RNN) consisting of encoder and decoder to capture long-term and current dynamics, using delay coordinates.
• Train the RNN offline in a three-stage process: initialize with a Conditional Restricted Boltzmann Machine, train one-step prediction, then train an N-step decoder.
• Predict z over the horizon with the RNN and solve the MPC problem via gradient-based optimization (e.g., BFGS) with backpropagation through time.
• Implement online updates by assimilating sensor data to refine the RNN, reducing reliance on full-state information.

Experimental results

Research questions

• RQ1 Can a DeepMPC framework using a deep RNN surrogate for control-relevant observables achieve real-time performance for complex, high-dimensional fluid flows?
• RQ2 How does online learning from sensor data affect prediction accuracy and control performance in varying dynamical regimes (laminar, quasi-periodic, and chaotic)?
• RQ3 What role do physical features like symmetry play in improving data efficiency and control robustness?
• RQ4 What is the impact of prediction horizon length and regularization on control effectiveness across different Reynolds numbers?

Key findings

• The DeepMPC approach achieves good control performance for flows around cylinders, tracking prescribed lift trajectories under various Reynolds numbers.
• For a single cylinder at Re = 100, the method yields effective periodic control that suppresses natural vortex shedding.
• In the fluidic pinball configuration (three cylinders), tracking errors increase with Reynolds number (Re = 100, 140, 200) due to increasing complexity and chaos.
• Using symmetry-based data augmentation significantly reduces tracking error at Re = 140, nearly halving the mean and improving control of the second lift signal.
• Online learning with sensor data rapidly improves tracking and reduces control cost, demonstrating practical online adaptability in the feedback loop.
• The framework remains viable with limited sensors and scales to higher complexity by focusing on control-relevant quantities rather than full state, with potential for further robustness enhancements.