[Paper Review] A New Approach to Formal Language Theory by Kolmogorov Complexity

Ming Li, Paul Vitányi|ArXiv.org|Oct 18, 2001

Computability, Logic, AI Algorithms3 citations

TL;DR

This paper introduces a novel framework for formal language theory using Kolmogorov complexity, replacing traditional pumping lemmas with incompressibility arguments. It provides a new characterization of regular languages and a powerful, easy-to-apply criterion to separate deterministic from nondeterministic context-free languages, with applications extending to nonrecursive languages via complexity bounds.

ABSTRACT

We present a new approach to formal language theory using Kolmogorov complexity. The main results presented here are an alternative for pumping lemma(s), a new characterization for regular languages, and a new method to separate deterministic context-free languages and nondeterministic context-free languages. The use of the new `incompressibility arguments' is illustrated by many examples. The approach is also successful at the high end of the Chomsky hierarchy since one can quantify nonrecursiveness in terms of Kolmogorov complexity. (This is a preliminary uncorrected version. The final version is the one published in SIAM J. Comput., 24:2(1995), 398-410.)

Motivation & Objective

• To develop a new, more intuitive and powerful method for proving nonregularity in formal languages using Kolmogorov complexity.
• To replace complex, ad hoc pumping lemmas with a unified incompressibility-based characterization of regular languages.
• To provide a simple, general-purpose criterion for distinguishing deterministic from nondeterministic context-free languages.
• To extend the applicability of Kolmogorov complexity to higher levels of the Chomsky hierarchy, including nonrecursive and recursively enumerable languages.
• To demonstrate that incompressibility arguments can yield concise, effective proofs where traditional methods fail or become unwieldy.

Proposed method

• Using the incompressibility of most strings—those with high Kolmogorov complexity—as a proof technique to derive lower bounds on language complexity.
• Defining a language as regular if and only if the prefix of length $ n $ of its characteristic sequence has Kolmogorov complexity $ O(\log n) $, providing a new characterization.
• Applying the KC-DCFL Lemma, which gives necessary conditions for a language to be deterministic context-free based on prefix complexity bounds.
• Using the fact that a string cannot be compressed if it is incompressible, to derive contradictions when assuming a language has a certain complexity class.
• Constructing individual incompressible strings as witnesses in proofs, avoiding the need to consider all strings as in classical counting arguments.
• Employing prefix complexity $ K $ and plain complexity $ C $ to analyze the complexity of characteristic sequences of languages, particularly in the context of recursive and nonrecursive reals.

Experimental results

Research questions

• RQ1 Can Kolmogorov complexity provide a more general and intuitive alternative to pumping lemmas for proving nonregularity?
• RQ2 Is there a single, unified characterization of regular languages based on Kolmogorov complexity that subsumes all known pumping lemmas?
• RQ3 Can incompressibility arguments effectively separate deterministic and nondeterministic context-free languages, where previous methods were cumbersome?
• RQ4 Can Kolmogorov complexity be used to quantify nonrecursiveness in formal languages at the top of the Chomsky hierarchy?
• RQ5 What are the necessary complexity conditions for a language to be deterministic context-free, and how can they be applied practically?

Key findings

• The paper establishes a new characterization of regular languages: a language is regular if and only if the prefix of length $ n $ of its characteristic sequence has Kolmogorov complexity $ O(\log n) $.
• The KC-DCFL Lemma provides a necessary condition for a language to be deterministic context-free, based on the complexity of its prefixes, and is shown to be effective for separating languages where traditional methods fail.
• The method successfully handles examples of nonregular languages that require special-purpose pumping lemmas, demonstrating broader applicability than existing techniques.
• The paper proves that only finitely many infinite binary sequences $ \omega $ satisfy $ C(\omega_{1:n}) \leq \log n + c $ for all $ n $, implying that such sequences are rare and can be used to characterize complexity classes.
• For recursive languages, the paper gives a known characterization, and for recursively enumerable languages, it provides a necessary condition based on Kolmogorov complexity.
• The proof of Claim 1 shows that the set of sequences with $ C(\omega_{1:n}) \leq \log n + c $ for all $ n $ is finite, even without assuming recursiveness, using only complexity bounds and string reconstruction.