[Paper Review] Representing Knowledge as Predictions (and State as Knowledge)

Mark Ring|arXiv (Cornell University)|Dec 12, 2021

Philosophy and History of Science4 citations

TL;DR

This paper proposes a hierarchical knowledge representation framework where artificial agents learn abstract, high-level knowledge directly from raw sensorimotor experience using General Value Functions (GVFs), or 'forecasts.' By recursively predicting future states and features based on actions, the agent builds layered abstractions—from raw pixels and touch to concepts like walls, rooms, and doorways—enabling intuitive, experience-based reasoning without human-annotated symbols.

ABSTRACT

This paper shows how a single mechanism allows knowledge to be constructed layer by layer directly from an agent's raw sensorimotor stream. This mechanism, the General Value Function (GVF) or "forecast," captures high-level, abstract knowledge as a set of predictions about existing features and knowledge, based exclusively on the agent's low-level senses and actions. Thus, forecasts provide a representation for organizing raw sensorimotor data into useful abstractions over an unlimited number of layers--a long-sought goal of AI and cognitive science. The heart of this paper is a detailed thought experiment providing a concrete, step-by-step formal illustration of how an artificial agent can build true, useful, abstract knowledge from its raw sensorimotor experience alone. The knowledge is represented as a set of layered predictions (forecasts) about the agent's observed consequences of its actions. This illustration shows twelve separate layers: the lowest consisting of raw pixels, touch and force sensors, and a small number of actions; the higher layers increasing in abstraction, eventually resulting in rich knowledge about the agent's world, corresponding roughly to doorways, walls, rooms, and floor plans. I then argue that this general mechanism may allow the representation of a broad spectrum of everyday human knowledge.

Motivation & Objective

• To address the longstanding challenge in AI and cognitive science of connecting high-level knowledge to its real-world basis in sensorimotor experience.
• To demonstrate that abstract, useful knowledge—such as understanding object properties or spatial layouts—can emerge organically from low-level interactions without human-annotated symbols.
• To provide a formal, step-by-step mechanism for constructing layered knowledge representations through predictive modeling of future states.
• To argue that this framework can represent a broad spectrum of everyday human knowledge, including intuitive physical and spatial reasoning.
• To position this approach as a foundation for continual learning systems that develop knowledge incrementally through experience, akin to human development.

Proposed method

• Uses General Value Functions (GVFs), also called 'forecasts,' to represent knowledge as predictions about future features or states based on current observations and actions.
• Constructs knowledge in a hierarchical, layer-by-layer fashion: lower layers process raw sensory inputs (e.g., pixels, touch), while higher layers abstract to concepts like walls, rooms, and doorways.
• Employs a recursive temporal difference update mechanism (Equation 11) that allows learning under any policy, with corrections for off-policy data using importance sampling.
• Defines forecasts as value functions over state features, where the target is a weighted combination of immediate and future predictions: $ f(s) = \beta(s)z(s) + (1 - \beta(s))c(s) + \sum_{s'} P(s, s') f(s') $, enabling stable learning.
• Introduces a structured set of forecasts, options, and aliases (e.g., 'wall right forward', 'go to center of room') to build increasingly abstract representations from primitive actions.
• Uses a policy-agnostic learning framework that supports off-policy training, allowing agents to learn from diverse experiences even when the behavior policy differs from the target policy.

Experimental results

Research questions

• RQ1 Can high-level, abstract knowledge—such as understanding object properties or spatial layouts—be constructed directly from raw sensorimotor data without human-annotated symbols?
• RQ2 How can a single, unified mechanism (GVFs) support the construction of layered, hierarchical knowledge representations from low-level sensory inputs?
• RQ3 To what extent can predictive models of future states serve as a foundation for intuitive reasoning, such as estimating weight, size, or stability of objects?
• RQ4 Can this framework support continual learning, where knowledge accumulates and evolves through ongoing interaction with the environment?
• RQ5 How can off-policy learning be integrated into this framework to allow efficient and stable learning from diverse experience trajectories?

Key findings

• The framework successfully constructs 12 layers of abstraction—from raw sensor inputs to high-level spatial concepts like rooms and doorways—demonstrating a plausible path to human-like intuitive knowledge.
• The use of GVFs enables the agent to represent knowledge as predictions about future states, allowing it to reason about object properties (e.g., weight, softness) through imagined interactions.
• The recursive update rule (Equation 11) supports temporal difference learning under any policy, enabling stable and efficient learning even when behavior and target policies differ.
• The system supports off-policy learning via importance sampling corrections, making it practical for real-world deployment where experience is collected under different policies.
• The method allows for the emergence of complex, high-level concepts (e.g., 'center of room', 'middle of hallway') through the composition of simpler forecasts and options.
• The framework provides a mechanism for representing not just facts, but also intuitive, experiential knowledge—such as expectations about how a balloon might behave—without explicit symbolic encoding.