[Paper Review] Actively Learning Gaussian Process Dynamics

Mona Buisson-Fenet, Friedrich Solowjow+1 more|arXiv (Cornell University)|Jul 31, 2020

Gaussian Processes and Bayesian Inference24 citations

TL;DR

This paper proposes active learning strategies for Gaussian process dynamics that select sampling points based on information-theoretic uncertainty, enabling sample-efficient model training while respecting system dynamics constraints. The approach achieves improved data efficiency through targeted exploration in high-uncertainty regions, validated via extensive numerical benchmarks.

ABSTRACT

Despite the availability of ever more data enabled through modern sensor and computer technology, it still remains an open problem to learn dynamical systems in a sample-efficient way. We propose active learning strategies that leverage information-theoretical properties arising naturally during Gaussian process regression, while respecting constraints on the sampling process imposed by the system dynamics. Sample points are selected in regions with high uncertainty, leading to exploratory behavior and data-efficient training of the model. All results are finally verified in an extensive numerical benchmark.

Motivation & Objective

• Address the challenge of sample inefficiency in learning dynamical systems despite increasing data availability.
• Develop active learning strategies that exploit information-theoretic properties inherent in Gaussian process regression.
• Ensure sampling constraints are respected during the learning process to remain compatible with real-world system dynamics.
• Achieve data-efficient model training by focusing on regions of high uncertainty for exploratory data collection.

Proposed method

• Use information-theoretic criteria to quantify uncertainty in Gaussian process regression for dynamic systems.
• Select new training points in regions of high predictive variance to guide exploratory data collection.
• Integrate system dynamics constraints into the active learning selection process to ensure feasible sampling.
• Leverage the natural uncertainty estimates from GP regression to drive adaptive sampling without additional modeling.
• Apply the strategy in a sequential learning framework where each new sample improves model accuracy and reduces uncertainty.

Experimental results

Research questions

• RQ1 How can active learning be effectively applied to Gaussian process models of dynamical systems to improve sample efficiency?
• RQ2 What information-theoretic criteria can guide the selection of informative sampling points in dynamic systems?
• RQ3 How can system dynamics constraints be integrated into active learning without compromising exploration?
• RQ4 To what extent does uncertainty-based sampling reduce the number of required training samples compared to random or uniform sampling?

Key findings

• The proposed active learning strategy significantly improves data efficiency by focusing on high-uncertainty regions.
• Uncertainty-driven sampling leads to faster convergence and reduced model error compared to baseline sampling methods.
• The method respects system dynamics constraints, ensuring feasible and safe data collection in real-world applications.
• Extensive numerical benchmarks confirm the effectiveness and robustness of the approach across diverse dynamical systems.