[Paper Review] Rational stochastic languages

François Denis, Yann Esposito|arXiv (Cornell University)|Feb 27, 2006

semigroups and automata theory14 references4 citations

TL;DR

This paper systematically studies rational stochastic languages—probability distributions over free monoids generated by multiplicity automata with parameters in semirings such as ℚ, ℝ, ℚ⁺, or ℝ⁺. It establishes that rational stochastic languages over ℝ or ℚ have finitely generated residual subsemimodules, enabling a structural link to probabilistic automata, while those over ℝ⁺ or ℚ⁺ may not, explaining the limitations of general probabilistic automata in inference and the success of probabilistic deterministic and residual automata.

ABSTRACT

The goal of the present paper is to provide a systematic and comprehensive study of rational stochastic languages over a semiring K \in {Q, Q +, R, R+}. A rational stochastic language is a probability distribution over a free monoid Σ^* which is rational over K, that is which can be generated by a multiplicity automata with parameters in K. We study the relations between the classes of rational stochastic languages S rat K (Σ). We define the notion of residual of a stochastic language and we use it to investigate properties of several subclasses of rational stochastic languages. Lastly, we study the representation of rational stochastic languages by means of multiplicity automata.

Motivation & Objective

• To provide a comprehensive theoretical framework for rational stochastic languages over various semirings, including ℚ, ℝ, ℚ⁺, and ℝ⁺.
• To clarify the relationship between rational stochastic languages and probabilistic automata, especially in the context of grammatical inference.
• To investigate the role of residual languages (derivatives) in characterizing rationality and finiteness of representations.
• To determine whether rational stochastic languages with rational values can be represented using coefficients in ℚ⁺ or ℝ⁺, and whether such representations are always possible.

Proposed method

• Uses formal power series over semirings K ∈ {ℚ, ℝ, ℚ⁺, ℝ⁺} to define rational stochastic languages as those representable by multiplicity automata with coefficients in K.
• Introduces the notion of residual language u⁻¹p(w) = p(uw)/p(uΣ*) for words u with non-zero probability, showing residuals are themselves stochastic languages.
• Defines the residual subsemimodule [Res(p)] as the span of all residual languages of a stochastic language p, and proves that p is rational iff [Res(p)] is finitely generated.
• Applies the theory of formal power series and semimodules to analyze closure properties and inclusion relations between classes of rational stochastic languages.
• Constructs a multiplicity automaton from a finite set of residual languages to show equivalence with probabilistic deterministic automata (PDA) when K ∈ {ℝ⁺, ℚ⁺}.
• Uses the concept of Fatou extensions to compare rationality over different semirings, showing ℝ is a Fatou extension of ℚ but ℝ⁺ is not of ℚ⁺.

Experimental results

Research questions

• RQ1 Is every rational stochastic language over ℝ⁺ that takes values in ℚ⁺ also rational over ℚ⁺?
• RQ2 Can a rational stochastic language with rational values always be represented by a multiplicity automaton with coefficients in ℚ⁺?
• RQ3 Does the residual subsemimodule of a rational stochastic language over ℝ or ℚ always admit a finite basis of residual languages?
• RQ4 What is the relationship between the classes of rational stochastic languages over different semirings, particularly in terms of inclusion and closure?
• RQ5 Under what conditions does a rational stochastic language over ℝ⁺ or ℚ⁺ admit a finite representation via residual languages?

Key findings

• The semiring ℝ is a Fatou extension of ℚ for stochastic languages: any rational stochastic language over ℝ with values in ℚ is also rational over ℚ.
• The semiring ℝ⁺ is not a Fatou extension of ℚ⁺: there exist rational stochastic languages over ℝ⁺ with values in ℚ⁺ that are not rational over ℚ⁺.
• The residual subsemimodule [Res(p)] of a rational stochastic language p is finitely generated if and only if p is rational, and this holds for K = ℝ or ℚ.
• For K ∈ {ℝ⁺, ℚ⁺}, a rational stochastic language has finitely many residual languages if and only if it is generated by a probabilistic deterministic automaton (PDA).
• The class of rational stochastic languages over ℝ⁺ equals the class of languages generated by probabilistic automata (PA), and similarly for ℚ⁺.
• The class of rational stochastic languages over ℝ (or ℚ) equals the class of languages generated by probabilistic residual automata (PRA), and this class is also equal to the class of languages with finitely generated residual subsemimodules.