[Paper Review] Learning For Predictive Control: A Dual Gaussian Process Approach

Yuhan Liu, Pengyu Wang+1 more|arXiv (Cornell University)|Nov 7, 2022

Advanced Control Systems Optimization4 citations

TL;DR

This paper proposes a Dual Gaussian Process (DGP) model predictive control (MPC) framework that combines a long-term GP for preserving historical knowledge and a short-term GP for real-time online adaptation to unknown or time-varying dynamics. The method enables efficient, forgetting-resistant online learning with recursive updates, significantly improving control performance in repetitive tasks compared to baseline and single-GP MPC approaches, achieving up to 90% reduction in tracking error after one iteration.

ABSTRACT

An important issue in model-based control design is that an accurate dynamic model of the system is generally nonlinear, complex, and costly to obtain. This limits achievable control performance in practice. Gaussian process (GP) based estimation of system models is an effective tool to learn unknown dynamics directly from input/output data. However, conventional GP-based control methods often ignore the computational cost associated with accumulating data during the operation of the system and how to handle forgetting in continuous adaption. In this paper, we present a novel Dual Gaussian Process (DGP) based model predictive control (MPC) strategy that enables efficient use of online learning based predictive control without the danger of catastrophic forgetting. The bio-inspired DGP structure is a combination of a long-term GP and a short-term GP, where the long-term GP is used to keep the learned knowledge in memory and the short-term GP is employed to rapidly compensate unknown dynamics during online operation. Furthermore, a novel recursive online update strategy for the short-term GP is proposed to successively improve the learnt model during online operation. Effectiveness of the proposed strategy is demonstrated via numerical simulations.

Motivation & Objective

• To address the challenge of catastrophic forgetting in online GP-based model predictive control.
• To enable efficient, recursive online learning of time-varying system uncertainties without data bloating.
• To improve control performance in repetitive tasks by preserving learned knowledge while adapting to new dynamics.
• To develop a scalable, computationally efficient GP-MPC framework suitable for real-time implementation.
• To demonstrate the superiority of dual GP structure over single-GP and baseline MPC in handling unknown and time-varying dynamics.

Proposed method

• A dual GP architecture is designed with a long-term GP (LGP) to store historical knowledge from offline training and a short-term GP (SGP) for online adaptation.
• The SGP is recursively updated using a novel online update strategy that maintains model accuracy under streaming data.
• The SGP compensates for unmodeled or time-varying dynamics during online operation, while the LGP preserves previously learned dynamics.
• The overall posterior distribution combines both GPs, ensuring stability and robustness by leveraging long-term memory and short-term learning.
• The method is integrated into a model predictive control framework, with uncertainty propagation handled via probabilistic GP predictions.
• A recursive update rule is derived to efficiently update the SGP posterior without retraining from scratch, reducing computational cost.

Experimental results

Research questions

• RQ1 Can a dual GP structure effectively prevent catastrophic forgetting in online GP-based control?
• RQ2 How does the combination of long-term memory and short-term adaptation improve control performance in repetitive tasks?
• RQ3 What is the impact of recursive online updating on the accuracy and convergence of the GP model in dynamic environments?
• RQ4 How does the proposed DGP-MPC compare to single-GP and baseline MPC in handling time-varying disturbances?
• RQ5 Can the DGP structure be effectively applied to online learning-based MPC with minimal computational overhead?

Key findings

• The DGP-MPC achieved a tracking error of 0.07×10⁻³ in the X-axis after the second iteration, representing a 90% reduction compared to the baseline MPC (3.30×10⁻³).
• In the Y-axis, the DGP-MPC reduced error from 7.71×10⁻³ (baseline) to 0.04×10⁻³ after two iterations, demonstrating strong learning efficiency.
• The OGP-MPC showed better performance than DGP-MPC in the first 10 seconds due to faster adaptation, but DGP-MPC outperformed OGP in long-term stability and memory retention.
• The DGP structure maintained lower estimation error (MSE) than OGP and LGP across both constant and time-varying wind phases, especially in the first half of the task.
• The DGP-MPC achieved superior performance in repetitive tasks by preserving historical data, while OGP-MPC suffered from knowledge decay due to continuous posterior updates.
• The proposed recursive update strategy enabled effective online learning without data bloating, maintaining computational efficiency and model accuracy.