[Paper Review] Short lists with short programs for functions

Nikolay Vereshchagin|arXiv (Cornell University)|Sep 20, 2014

Computability, Logic, AI Algorithms1 references3 citations

TL;DR

This paper establishes a fundamental lower bound on the size of shortlists containing near-optimal programs for computable functions. It proves that for any total computable function mapping programs to lists, there are infinitely many programs for which the list must contain exponentially many strings—specifically, at least $2^{|p| - \varepsilon(p) - O(1)}$ strings—to include a program for the function that is only $\varepsilon(p)$ bits longer than the shortest possible.

ABSTRACT

Let $\{ϕ_p\}$ be an optimal Gödel numbering of the family of computable functions (in Schnorr's sense), where $p$ ranges over binary strings. Assume that a list of strings $L(p)$ is computable from $p$ and for all $p$ contains a $ϕ$-program for $ϕ_p$ whose length is at most $\varepsilon$ bits larger that the length of the shortest $ϕ$-program for $ϕ_p$. We show that for infinitely many $p$ the list $L(p)$ must have $2^{|p|-\varepsilon-O(1)}$ strings. Here $\varepsilon$ is an arbitrary function of $p$.

Motivation & Objective

• To resolve a question posed by Alexander Shen regarding the existence of a total computable algorithm that, given any program for a string, outputs a shortlist containing a near-optimal program for that string.
• To establish tight lower bounds on the size of such shortlists when the list must contain a program whose length is within $\varepsilon(p)$ bits of the minimal possible.
• To demonstrate that no such total computable shortlist construction can be efficient—specifically, that list sizes must grow exponentially in the input length minus the excess length $\varepsilon(p)$.

Proposed method

• Constructs a universal machine $U$ and a computable function $V$ that maps programs to strings, using a diagonalization argument to ensure high Kolmogorov complexity for target strings.
• Employs a recursive enumeration of the graphs of universal machines $U$ and $\Phi$ (a universal machine for lists) to maintain partial approximations $\bar{U}$ and $\bar{\Phi}$.
• Uses a condition-checking mechanism to detect when the current state of approximations fails to satisfy required complexity and list size constraints.
• When such failure occurs, dynamically reassigns program and string pairs ($q_{kn}, x_{kn}$) to restore the required properties, ensuring the existence of strings with high complexity and large required list sizes.
• Applies a counting argument to show that there are always unused strings of appropriate complexity and list compatibility to maintain the construction.
• Relies on the existence of a linearly bounded translator from $V$ to $U$ to relate the program length $|p|$ to the length of the $V$-program $q$, preserving the exponential lower bound.

Experimental results

Research questions

• RQ1 Is there a total computable function that, given any $U$-program $p$ for a string $x$, outputs a shortlist $L(p)$ containing a $U$-program for $x$ of length at most $C_U(x) + \varepsilon(p)$?
• RQ2 What is the minimal size of such a shortlist $L(p)$, given that it must contain a program within $\varepsilon(p)$ bits of optimality?
• RQ3 Can the size of $L(p)$ be bounded by a polynomial in $|x|$ or $|p|$ while still guaranteeing near-optimal program inclusion?
• RQ4 How does the size of $L(p)$ relate to the excess length $\varepsilon(p) = C_{U,L(p)}(U(p)) - |U(p)|$?
• RQ5 Are there inherent limitations on the efficiency of shortlist construction that make exponential list sizes unavoidable for infinitely many inputs?

Key findings

• For any total computable function $L$ mapping programs to finite lists, there are infinitely many programs $p$ such that $L(p)$ must contain at least $2^{|x| - \varepsilon(p) - O(1)}$ strings to include a $U$-program for $x$ of length at most $C_U(x) + \varepsilon(p)$.
• The size of $L(p)$ is at least $2^{|p| - \varepsilon(p) - O(1)} - 2$ for infinitely many $p$, where $\varepsilon(p) = C_{U,L(p)}(U(p)) - |U(p)|$.
• The lower bound is tight in the sense that for certain computable lists (e.g., $L_i(p) = \{p\} \cup \{0,1\}^{<i}$), both the list size and the program length in the list match the theoretical bounds up to additive constants.
• The result holds even when $\varepsilon(p)$ is defined as $C_{U,L(p)}(U(p)) - C_U(U(p))$, showing the bound applies to the standard Kolmogorov complexity difference.
• The construction is effective: a recursive algorithm can be designed to maintain the required properties during enumeration, ensuring the existence of such strings and programs.
• The exponential lower bound arises from a counting argument: the number of strings of length $n$ with high Kolmogorov complexity exceeds the number of strings that can be covered by small lists with low description complexity.