[Paper Review] Open the Chests: An Environment for Activity Recognition and Sequential Decision Problems Using Temporal Logic

Benjamin Beyret, José Hernández‐Orallo+4 more|arXiv (Cornell University)|Sep 12, 2019

Reinforcement Learning in Robotics27 references35 citations

TL;DR

This paper introduces the Animal-AI environment, a physics-based, open-source platform designed to evaluate artificial agents on animal-like cognitive abilities such as object permanence, causal reasoning, and spatial reasoning. Inspired by comparative psychology, it uses structured tasks grounded in temporal logic and physical simulation to test generalization and understanding, achieving state-of-the-art AI performance around 40% across 10 cognitive categories.

ABSTRACT

Recent advances in artificial intelligence have been strongly driven by the use of game environments for training and evaluating agents. Games are often accessible and versatile, with well-defined state-transitions and goals allowing for intensive training and experimentation. However, agents trained in a particular environment are usually tested on the same or slightly varied distributions, and solutions do not necessarily imply any understanding. If we want AI systems that can model and understand their environment, we need environments that explicitly test for this. Inspired by the extensive literature on animal cognition, we present an environment that keeps all the positive elements of standard gaming environments, but is explicitly designed for the testing of animal-like artificial cognition.

Motivation & Objective

• To address the lack of ability-oriented benchmarks in AI by creating a testbed that evaluates generalization and understanding, not just task-specific performance.
• To bridge the gap between AI evaluation and animal cognition research by adapting established experimental paradigms from comparative psychology.
• To design a training and testing environment that emphasizes cognitive reasoning, physical understanding, and long-term memory over pattern recognition.
• To provide a scalable, open platform for evaluating agents across diverse cognitive skills using a unified, modular framework.
• To foster collaboration between AI and comparative cognition communities by enabling disentanglement of cognitive capabilities.

Proposed method

• The environment is built on a simulated physics engine with a deterministic state transition function, enabling reproducible and complex interactions.
• It features seven object classes, including rewards (food), blocks, and obstacles, allowing construction of diverse cognitive tasks.
• Tasks are organized into 10 cognitive categories—e.g., preferences, object permanence, causal reasoning—each testing a distinct cognitive ability.
• Each task is designed to be solvable by animals but challenging for current AI, using temporal logic to define goal conditions.
• The platform supports zero-shot generalization by testing agents on unseen task variations, with no retraining allowed.
• The Animal-AI Olympics competition uses this testbed to evaluate agents across all 10 categories, generating performance profiles beyond single scores.

Experimental results

Research questions

• RQ1 Can AI agents achieve human-level performance on tasks that require intuitive physics, object permanence, and causal reasoning, as seen in animal cognition?
• RQ2 To what extent can a single agent generalize across diverse, non-identical cognitive tasks without retraining?
• RQ3 How do current deep reinforcement learning methods perform on ability-oriented benchmarks compared to task-oriented ones?
• RQ4 Can structured, psychology-inspired tasks reveal fundamental limitations in current AI generalization and reasoning capabilities?
• RQ5 How can cognitive profiling of agents be achieved through a modular, multi-category benchmark?

Key findings

• Current state-of-the-art AI performance on the Animal-AI testbed is approximately 40%, indicating significant room for improvement in cognitive generalization.
• Navigation-based and reward-choice tasks are easier to solve, with performance approaching 80% in some cases, aligning with traditional reinforcement learning strengths.
• Tasks requiring internal world modeling, object permanence, and causal reasoning are significantly harder, with performance below 30% in the most complex variants.
• The radar plot of top competitors shows clear performance disparities across categories, with strong performance in preferences and obstacles, but weak performance in internal models and causal reasoning.
• The platform successfully identifies that agents trained on standard DRL benchmarks fail to generalize to tasks requiring long-term memory or physical reasoning.
• The results suggest that current AI systems lack robust internal models of the environment and struggle with reasoning about unseen or hidden states, mirroring known limitations in animal cognition.