[Paper Review] Computational Dualism and Objective Superintelligence

Michael Timothy Bennett|arXiv (Cornell University)|Feb 2, 2023

Computability, Logic, AI Algorithms23 references13 citations

TL;DR

The paper argues that AIXI’s performance is interpreter-dependent and proposes an enactive, pancomputational framework to define objectively optimal AGI/ASI by treating cognition as enacted within the environment and using a weakness-based proxy for intelligence.

ABSTRACT

The concept of intelligent software is flawed. The behaviour of software is determined by the hardware that "interprets" it. This undermines claims regarding the behaviour of theorised, software superintelligence. Here we characterise this problem as "computational dualism", where instead of mental and physical substance, we have software and hardware. We argue that to make objective claims regarding performance we must avoid computational dualism. We propose a pancomputational alternative wherein every aspect of the environment is a relation between irreducible states. We formalise systems as behaviour (inputs and outputs), and cognition as embodied, embedded, extended and enactive. The result is cognition formalised as a part of the environment, rather than as a disembodied policy interacting with the environment through an interpreter. This allows us to make objective claims regarding intelligence, which we argue is the ability to "generalise", identify causes and adapt. We then establish objective upper bounds for intelligent behaviour. This suggests AGI will be safer, but more limited, than theorised.

Motivation & Objective

• Explain why AIXI’s performance is subjective to the interpreter (UTM) used.
• Propose enactive cognition and pancomputationalism as a framework to remove mind–body dualism in AI.
• Introduce a weakness-based proxy for intelligence to evaluate generalisation across tasks.
• Define objectively optimal AGI as the best hypothesis for a given task within a chosen vocabulary.
• Define objectively optimal ASI as selecting the optimal vocabulary to maximize intelligence for a task and then constructing an AGI.

Proposed method

• Formalise cognition as a task that merges agent, body, and environment (enactivism/pancomputationalism).
• Model the environment within the environment by treating the task as the primary object of modelling.
• Define implementable languages (vocabularies) and a finite sensorimotor vocabulary for enactment.
• Introduce a binary task completion criterion and a set of declarative programs to describe tasks and decisions.
• Use a weakness-based proxy (size of the extension Z_m) to select the optimal hypothesis.
• Propose definitions of objectively optimal AGI and ASI based on maximal weakness and task-generalisation properties.

Experimental results

Research questions

• RQ1 How can cognition be formalised to be independent of the interpreter (UTM) in evaluating AI performance?
• RQ2 How can we define AGI and ASI objectively using enactive cognition and pancomputationalism?
• RQ3 What proxy best measures intelligence if description length is inappropriate, and how does weakness serve this role?
• RQ4 How can tasks be defined and evaluated within a finite implementable vocabulary rather than entire environments?
• RQ5 How should performance be measured in a finite, task-focused setting to reflect generalisation speed?

Key findings

• AIXI’s apparent Pareto optimality is contingent on the interpreter; fixing the interpreter can mitigate subjectivity but raises other issues.
• An enactive, pancomputational framework can integrate mind and environment, treating cognition as enacted within a local task-centric vocabulary.
• Weakness (the size of a model’s extension) is argued as a superior proxy for generalisation and intelligence than description length.
• A formal notion of a v-task (vocabulary-based task) is introduced to replace environment-centric modelling with task-centric modelling.
• Objectively optimal AGI is the hypothesis with maximal weakness within the task vocabulary, and ASI selects the vocabulary that maximises intelligence value for a given task.
• The framework provides a pathway to define, compare, and potentially engineer AGI/ASI systems whose performance is intrinsic to the task rather than the interpreter.