[Paper Review] On the Running Time of the Shortest Programs

Norbert Bátfai|ArXiv.org|Aug 10, 2009

Computability, Logic, AI Algorithms5 references4 citations

TL;DR

This paper investigates the running time of the shortest programs that generate a given string, introducing Kolmogorov time complexity $ K_T(w) $ as the minimal runtime among all shortest programs for string $ w $. It establishes that for incompressible strings, $ K_T(w) = O(1) $, and analyzes Busy Beaver and Placid Platypus machines to explore extreme runtime behaviors, showing that even highly compressible strings can have short-running shortest programs.

ABSTRACT

The Kolmogorov complexity of the word w is equal to the length of the shortest concatenation of program Z and its input x with which the word w is computed by the universal turing machine U. The question introduced in this paper is the following: How long do the shortest programs run for?

Motivation & Objective

• To understand the runtime behavior of the shortest programs that generate a given string, beyond just their length.
• To define and analyze a new complexity measure: Kolmogorov time complexity $ K_T(w) $, the minimal runtime among all shortest programs for $ w $.
• To investigate whether the shortest programs for highly compressible strings (like Busy Beaver words) can run for extremely long times.
• To explore the relationship between Kolmogorov complexity and runtime by studying concrete machines, particularly Busy Beaver and Placid Platypus machines.
• To provide empirical evidence and theoretical bounds on $ K_T(w) $, especially for incompressible and compressible strings.

Proposed method

• Defines $ K_T(w) = \min\{ t_U(Zx) \mid U(Zx) = w, \, |Zx| = K(w) \} $, the minimal runtime of any shortest program for $ w $.
• Uses the universal Turing machine $ U $ to simulate programs $ Zx $, measuring runtime $ t_U(Zx) $ in steps.
• Applies bounds on universal machine overhead: $ t_U(Zx) \leq c_{\text{init}}|Zx| + c_{\text{run}} t_Z(|x|) $, showing $ K_t(w) \leq c|w| $.
• Analyzes specific Busy Beaver machines $ \mathcal{M}_{BB}(n) $ for $ n = 1 $ to $ 4 $, computing their runtime and output.
• Employs custom C programs to search for and verify Placid Platypus machines, which produce many 1s with few states and rules.
• Uses a rule indexing system in C code to represent and simulate Turing machine transitions efficiently.

Experimental results

Research questions

• RQ1 What is the minimal runtime among all shortest programs that generate a given string $ w $?
• RQ2 Can the shortest program for an incompressible string $ w $ run for a long time, or is its runtime bounded?
• RQ3 How does the runtime of the shortest program for a highly compressible string (like a Busy Beaver word) compare to its length?
• RQ4 Are there theoretical upper or lower bounds on $ K_T(w) $, especially for strings with $ K(w) \approx |w| $?
• RQ5 Can the runtime of the shortest program be characterized independently of its length, particularly in the context of undecidability?

Key findings

• For incompressible strings where $ K(w) = |w| + |Z_{\text{trivial}}| $, the runtime of the shortest program is bounded by $ O(1) $, as shown by $ K_T(w) \leq c_{\text{init}}|Z_{\text{trivial}}||w| $.
• The paper confirms that the shortest program for an incompressible string runs in constant time relative to the string length, implying $ K_T(w) = O(1) $.
• For $ n=4 $, the Busy Beaver machine $ \mathcal{M}_{BB}(4) $ produces $ 1^{12} $ and runs for 95 to 106 steps, showing that even short programs can have non-trivial runtimes.
• The Placid Platypus machine with 5 states and 9 rules produces $ 1^{160} $ in 20,927 steps, demonstrating that short programs can generate very long outputs in long runtimes.
• Empirical results show that the shortest programs for highly compressible strings (like $ 1^{12} $) can have runtimes significantly longer than the string length, but still bounded by universal machine overhead.
• The study confirms that the shortest program for a string cannot run indefinitely, supporting the conjecture that shortest programs always halt and have finite, bounded runtimes.