[Paper Review] Learning particle swarming models from data with Gaussian processes

Jinchao Feng, Kulick, Charles+2 more|arXiv (Cornell University)|Jun 4, 2021

Gaussian Processes and Bayesian Inference53 references4 citations

TL;DR

This paper proposes a Gaussian process-based method to learn second-order particle swarming models from sparse, noisy trajectory data, enabling nonparametric inference of radial interaction kernels and parametric estimation of friction forces. It establishes optimal convergence rates in Reproducing Kernel Hilbert Space and provides uncertainty quantification, with theoretical guarantees under a coercivity condition for recoverability.

ABSTRACT

Interacting particle or agent systems that display a rich variety of swarming behaviours are ubiquitous in science and engineering. A fundamental and challenging goal is to understand the link between individual interaction rules and swarming. In this paper, we study the data-driven discovery of a second-order particle swarming model that describes the evolution of $N$ particles in $\mathbb{R}^d$ under radial interactions. We propose a learning approach that models the latent radial interaction function as Gaussian processes, which can simultaneously fulfill two inference goals: one is the nonparametric inference of {the} interaction function with pointwise uncertainty quantification, and the other one is the inference of unknown scalar parameters in the non-collective friction forces of the system. We formulate the learning problem as a statistical inverse problem and provide a detailed analysis of recoverability conditions, establishing that a coercivity condition is sufficient for recoverability. Given data collected from $M$ i.i.d trajectories with independent Gaussian observational noise, we provide a finite-sample analysis, showing that our posterior mean estimator converges in a Reproducing kernel Hilbert space norm, at an optimal rate in $M$ equal to the one in the classical 1-dimensional Kernel Ridge regression. As a byproduct, we show we can obtain a parametric learning rate in $M$ for the posterior marginal variance using $L^{\infty}$ norm, and the rate could also involve $N$ and $L$ (the number of observation time instances for each trajectory), depending on the condition number of the inverse problem. Numerical results on systems that exhibit different swarming behaviors demonstrate efficient learning of our approach from scarce noisy trajectory data.

Motivation & Objective

• To address the inverse problem of discovering interaction laws in second-order particle systems from limited, noisy trajectory data.
• To enable nonparametric inference of radial interaction kernels without assuming specific functional forms.
• To simultaneously estimate unknown scalar parameters in non-collective friction forces.
• To provide rigorous theoretical guarantees on recoverability and convergence rates under minimal assumptions.
• To deliver pointwise uncertainty quantification for the inferred interaction functions.

Proposed method

• Model the latent radial interaction function as a Gaussian process prior to enable nonparametric inference with uncertainty quantification.
• Formulate the learning problem as a statistical inverse problem using an operator-theoretic framework.
• Use a coercivity condition as a sufficient condition for the recoverability of the interaction kernel.
• Derive a posterior mean estimator that converges at the optimal rate in the Reproducing Kernel Hilbert Space norm, matching classical 1D Kernel Ridge Regression.
• Analyze the posterior marginal variance using the $L^\infty$ norm, showing parametric convergence rates in $M$ that may depend on $N$ and $L$ via the condition number.
• Apply the framework to systems with diverse swarming behaviors, demonstrating robustness in low-data, noisy regimes.

Experimental results

Research questions

• RQ1 Can we nonparametrically infer the radial interaction kernel in second-order particle swarming systems from sparse, noisy trajectory data?
• RQ2 What conditions ensure the theoretical recoverability of the interaction function in this inverse problem?
• RQ3 How fast does the posterior mean estimator converge, and does it achieve the optimal rate in finite-sample settings?
• RQ4 Can we obtain reliable uncertainty quantification for the inferred interaction kernel and friction parameters?
• RQ5 How do the number of trajectories $M$, particles $N$, and observation times $L$ affect the learning accuracy and variance?

Key findings

• The posterior mean estimator converges in the Reproducing Kernel Hilbert Space norm at the optimal rate in $M$, matching the classical 1D Kernel Ridge Regression rate.
• The posterior marginal variance converges at a parametric rate in $M$ under the $L^\infty$ norm, with potential dependence on $N$ and $L$ through the condition number of the inverse problem.
• A coercivity condition is sufficient to ensure the recoverability of the interaction kernel, providing a theoretical foundation for identifiability.
• Numerical experiments confirm the method’s effectiveness in learning diverse swarming behaviors from scarce and noisy trajectory data.
• The approach delivers pointwise uncertainty quantification for the interaction kernel, enabling confidence assessment in learned dynamics.
• The framework successfully infers both the nonparametric interaction function and unknown scalar friction parameters in a unified Bayesian setting.