[Paper Review] Data-Driven Stochastic Reachability Using Hilbert Space Embeddings.

Adam J. Thorpe, Kendric R. Ortiz+1 more|arXiv (Cornell University)|Oct 15, 2020

Adversarial Robustness in Machine Learning52 references4 citations

TL;DR

This paper proposes a data-driven approach to stochastic reachability using Hilbert space embeddings of probability distributions, enabling finite-sample probabilistic guarantees on safety for systems with arbitrary disturbances. By embedding the stochastic kernel in a reproducing kernel Hilbert space, the method computes safety probabilities via matrix operations and inner products, with theoretical bounds derived from statistical learning theory, validated on a neural-net-controlled pendulum system.

ABSTRACT

We compute finite sample bounds for approximations of the solution to stochastic reachability problems computed using kernel distribution embeddings, a non-parametric machine learning technique. Our approach enables assurances of safety from observed data, through construction of probabilistic violation bounds on the computed stochastic reachability probability. By embedding the stochastic kernel of a Markov control process in a reproducing kernel Hilbert space, we can compute the safety probabilities for systems with arbitrary disturbances as simple matrix operations and inner products. We present finite sample bounds for the approximation using elements from statistical learning theory. We numerically evaluate the approach, and demonstrate its efficacy on neural net-controlled pendulum system.

Motivation & Objective

• To enable safety verification for stochastic control systems using only observed data, without parametric assumptions.
• To address the challenge of computing reliable reachability probabilities when system disturbances are arbitrary and unknown.
• To provide finite-sample probabilistic bounds on the approximation error of reachability probabilities derived from empirical data.
• To develop a computationally efficient framework based on kernel methods that transforms complex stochastic dynamics into matrix operations.

Proposed method

• Embed the stochastic kernel of a Markov control process into a reproducing kernel Hilbert space (RKHS) using kernel distribution embeddings.
• Represent the system's transition probabilities as elements in an RKHS, enabling non-parametric estimation from data.
• Compute reachability probabilities as inner products in the RKHS, reducing the problem to matrix operations on empirical data.
• Derive finite-sample bounds on the approximation error using tools from statistical learning theory, particularly covering number-based generalization bounds.
• Apply the method to a neural network-controlled pendulum system to evaluate performance and robustness.
• Ensure theoretical safety guarantees by constructing probabilistic violation bounds on the computed reachability probabilities.

Experimental results

Research questions

• RQ1 Can kernel-based Hilbert space embeddings provide reliable, data-driven approximations of stochastic reachability probabilities with finite-sample guarantees?
• RQ2 How can the approximation error of reachability probabilities be bounded when using empirical data and non-parametric kernel methods?
• RQ3 To what extent can this method handle arbitrary, non-parametric disturbances in stochastic control systems?
• RQ4 How does the method scale and perform in a high-dimensional, nonlinear system such as a neural-net-controlled pendulum?
• RQ5 Can the approach provide probabilistic safety assurances without assuming a specific parametric form for the system's noise or transition dynamics?

Key findings

• The method achieves finite-sample probabilistic bounds on the error of computed reachability probabilities, enabling rigorous safety assurances from empirical data.
• Stochastic reachability probabilities are computed efficiently using only matrix operations and inner products in the RKHS, avoiding complex numerical integration.
• The approach is applicable to systems with arbitrary disturbances, as it does not require parametric assumptions on the noise distribution.
• The numerical evaluation on a neural-net-controlled pendulum demonstrates the method’s efficacy and practical feasibility in nonlinear, high-dimensional settings.
• Theoretical guarantees are derived using covering number-based bounds from statistical learning theory, ensuring reliability even with limited data.