[Paper Review] Symbolic Control for Stochastic Systems via Parity Games.

Rupak Majumdar, Kaushik Mallik+2 more|arXiv (Cornell University)|Jan 4, 2021

Formal Methods in Verification37 references4 citations

TL;DR

This paper proposes a symbolic control method for stochastic nonlinear systems to compute a lower bound on the maximal probability of satisfying $ω$-regular specifications. By reducing the problem to a $2\frac{1}{2}$-player parity game via a finite-state abstraction that uses only the support of transition probabilities, it computes an under-approximation of the qualitative winning region and combines it with reachability synthesis to yield a provably correct lower bound on satisfaction probability, demonstrated on a perturbed Dubins vehicle model.

ABSTRACT

We consider the problem of computing the maximal probability of satisfying an $\omega$-regular specification for stochastic, continuous-state, nonlinear systems evolving in discrete time. The problem reduces, after automata-theoretic constructions, to finding the maximal probability of satisfying a parity condition on a (possibly hybrid) state space. While characterizing the exact satisfaction probability is open, we show that a lower bound on this probability can be obtained by (I) computing an under-approximation of the qualitative winning region, i.e., states from which the parity condition can be enforced almost surely, and (II) computing the maximal probability of reaching this qualitative winning region. The heart of our approach is a technique to symbolically compute the under-approximation of the qualitative winning region in step (I) via a finite-state abstraction of the original system as a $2\frac{1}{2}$-player parity game. Our abstraction procedure uses only the support of the probabilistic evolution; it does not use precise numerical transition probabilities. We prove that the winning set in the abstract $2\frac{1}{2}$-player game induces an under-approximation of the qualitative winning region in the original synthesis problem, along with a policy to solve it. By combining these contributions with (a) existing symbolic fixpoint algorithms to solve $2\frac{1}{2}$-player games and (b) existing techniques for reachability policy synthesis in stochastic nonlinear systems, we get an abstraction-based symbolic algorithm for finding a lower bound on the maximal satisfaction probability. We have implemented our approach and evaluated it on the nonlinear model of the perturbed Dubins vehicle.

Motivation & Objective

• To compute a lower bound on the maximal probability of satisfying $ω$-regular specifications in stochastic, continuous-state, nonlinear systems.
• To address the challenge of synthesizing control policies for such systems when exact satisfaction probabilities are intractable to compute.
• To develop a finite-state abstraction of the system that preserves qualitative winning regions using only the support of probabilistic transitions.
• To enable symbolic computation of control policies via $2\frac{1}{2}$-player parity games without requiring precise transition probabilities.
• To integrate symbolic fixpoint algorithms and reachability synthesis to yield a practical, scalable algorithm for stochastic nonlinear systems.

Proposed method

• Reduce the synthesis problem to a parity condition on a hybrid state space using automata-theoretic constructions.
• Construct a finite-state abstraction of the original system as a $2\frac{1}{2}$-player parity game based solely on the support of probabilistic transitions.
• Symbolically compute an under-approximation of the qualitative winning region in the abstract game, which induces a lower bound on the winning region in the original system.
• Leverage symbolic fixpoint algorithms to solve the abstract $2\frac{1}{2}$-player game and extract a control policy.
• Combine the under-approximated winning region with reachability policy synthesis to compute the maximal probability of reaching it.
• Use the resulting policy to derive a lower bound on the maximal satisfaction probability of the original specification.

Experimental results

Research questions

• RQ1 Can a lower bound on the maximal satisfaction probability of $ω$-regular specifications be computed for stochastic nonlinear systems without relying on precise transition probabilities?
• RQ2 How can a finite-state abstraction of a continuous-state stochastic system be constructed to preserve qualitative winning regions for parity conditions?
• RQ3 To what extent can symbolic methods based on $2\frac{1}{2}$-player parity games be used to synthesize control policies for nonlinear systems with stochastic dynamics?
• RQ4 What is the relationship between the winning set in the abstract game and the qualitative winning region in the original system?
• RQ5 Can the proposed abstraction-based approach be effectively implemented and evaluated on realistic nonlinear control models?

Key findings

• The proposed abstraction induces a provably correct under-approximation of the qualitative winning region in the original stochastic system.
• The method computes a lower bound on the maximal satisfaction probability by combining the under-approximated winning region with reachability probability analysis.
• The abstraction relies only on the support of the probabilistic transitions, avoiding the need for precise numerical probabilities.
• The symbolic fixpoint algorithms used to solve the abstract $2\frac{1}{2}$-player game are scalable and effective for the abstraction.
• The approach was successfully implemented and evaluated on a nonlinear model of the perturbed Dubins vehicle, demonstrating practical applicability.
• The resulting policy ensures almost-sure satisfaction of the parity condition from the under-approximated winning region, with a quantifiable lower bound on the overall satisfaction probability.