[Paper Review] A new rule for almost-certain termination of probabilistic- and demonic programs

Annabelle McIver, Carroll Morgan|arXiv (Cornell University)|Dec 4, 2016

Logic, Reasoning, and Knowledge13 references9 citations

TL;DR

This paper introduces a novel proof rule for establishing almost-certain termination (ACT) in probabilistic and demonic programs by combining super-martingales with a parametrized progress condition. The method enables verification of termination with probability one in infinite-state systems—such as the unbounded symmetric random walk in one and two dimensions—where prior techniques based on non-zero probability decreases or bounded super-martingales fail.

ABSTRACT

Extending our own and others' earlier approaches to reasoning about termination of probabilistic programs, we propose and prove a new rule for termination with probability one, also known as "almost-certain termination". The rule uses both (non-strict) super martingales and guarantees of progress, together, and it seems to cover significant cases that earlier methods do not. In particular, it suffices for termination of the unbounded symmetric random walk in both one- and two dimensions: for the first, we give a proof; for the second, we use a theorem of Foster to argue that a proof exists. Non-determinism (i.e. demonic choice) is supported; but we do currently restrict to discrete distributions.

Motivation & Objective

• To develop a proof rule for almost-certain termination (ACT) in probabilistic and demonic programs that works in infinite-state systems where traditional methods fail.
• To combine super-martingale reasoning with a local, parametrized progress condition to establish ACT without requiring probability bounds away from zero.
• To provide a source-code-level verification technique that avoids explicit construction of transition systems, enabling local reasoning over program code.
• To extend existing martingale-based methods to cover cases like the 2D symmetric random walk, which previous approaches could not verify directly.
• To establish a connection between boundedness of martingales and non-termination, offering a refutation certificate for programs that do not terminate with probability one.

Proposed method

• Define a non-negative variant function $ V $ over the state space $ S $, with $ V(s) = 0 $ for all terminal states $ s \in S_0 $, and $ V(s) > 0 $ for all non-terminal states $ s \in S_* $.
• Require that $ V $ be a super-martingale: the expected value of $ V $ after a transition is less than or equal to its current value, under any choice of distribution from the demonic set.
• Introduce a parametrized progress condition: for each state $ s \in S_* $, there exists a $ \delta > 0 $ such that with probability at least $ \delta $, the next state $ s' $ satisfies $ V(s') < V(s) $, and this $ \delta $ may depend on $ s $.
• Use the combination of super-martingale property and local progress to satisfy the conditions of a zero-one law, guaranteeing that the probability of eventual termination is one.
• Apply the rule to infinite-state systems such as the 1D and 2D symmetric random walks, where standard super-martingale rules fail due to lack of uniform lower bounds on transition probabilities.
• Leverage Foster’s theorem to argue existence of a proof for the 2D case, even when no closed-form variant is known.

Experimental results

Research questions

• RQ1 Can a unified proof rule be developed that combines super-martingales and progress conditions to verify almost-certain termination in infinite-state probabilistic and demonic programs?
• RQ2 Does the proposed rule extend the scope of verifiable programs beyond those covered by existing super-martingale-based methods, particularly for unbounded random walks?
• RQ3 Can the rule be applied directly to source code without constructing the underlying transition system explicitly?
• RQ4 What is the relationship between boundedness of martingales and the impossibility of almost-sure termination?
• RQ5 Can the rule serve as a refutation certificate for non-termination with probability one, not just for finite expected time?

Key findings

• The proposed rule successfully proves almost-certain termination for the unbounded symmetric random walk in one dimension, using a linear super-martingale and a local progress condition.
• For the two-dimensional symmetric random walk, although no closed-form variant is known, Foster’s theorem guarantees the existence of a variant, and the rule confirms ACT under the same conditions.
• The rule identifies a key distinction: if a martingale is bounded and non-constant, then almost-sure termination cannot hold—offering a refutation method for such cases.
• The method generalizes prior rules by combining super-martingales with parametrized progress, enabling verification in systems where transition probabilities are not uniformly bounded away from zero.
• The rule is sound and applicable to source-level reasoning, supporting integration into program logics like pGCL for formal verification.
• The paper establishes a new refutation certificate: a finite, non-constant exact martingale implies that a program does not terminate with probability one, not just that expected time is infinite.