[Paper Review] A Rubric for Human-like Agents and NeuroAI

Ida Momennejad|arXiv (Cornell University)|Dec 8, 2022

EEG and Brain-Computer Interfaces17 citations

TL;DR

This paper proposes a three-dimensional rubric to clarify the scope of human-like agents and NeuroAI research, distinguishing between commitments to human-like behavior, neural plausibility, and engineering benchmarks. By categorizing research along these axes, the framework enables clearer communication, identifies synergies, and promotes iterative, interdisciplinary progress across cognitive science, neuroscience, and AI.

ABSTRACT

Researchers across cognitive, neuro-, and computer sciences increasingly reference human-like artificial intelligence and neuroAI. However, the scope and use of the terms are often inconsistent. Contributed research ranges widely from mimicking behaviour, to testing machine learning methods as neurally plausible hypotheses at the cellular or functional levels, or solving engineering problems. However, it cannot be assumed nor expected that progress on one of these three goals will automatically translate to progress in others. Here a simple rubric is proposed to clarify the scope of individual contributions, grounded in their commitments to human-like behaviour, neural plausibility, or benchmark/engineering goals. This is clarified using examples of weak and strong neuroAI and human-like agents, and discussing the generative, corroborate, and corrective ways in which the three dimensions interact with one another. The author maintains that future progress in artificial intelligence will need strong interactions across the disciplines, with iterative feedback loops and meticulous validity tests, leading to both known and yet-unknown advances that may span decades to come.

Motivation & Objective

• To address the inconsistent and ambiguous use of terms like 'human-like agents' and 'NeuroAI' across cognitive, neuroscientific, and computer science research.
• To clarify the distinct goals of research in human-like behavior, neural plausibility, and engineering benchmarks.
• To provide a structured framework that distinguishes between weak and strong forms of NeuroAI and human-like agents.
• To promote interdisciplinary progress through iterative feedback and validity testing across the three dimensions.
• To guide future research by emphasizing the non-transferable nature of advances across the three domains, requiring deliberate integration.

Proposed method

• Proposes a three-axis rubric: (1) human-like behavior, (2) neural plausibility, and (3) benchmark/engineering goals.
• Uses the rubric to classify existing research into weak or strong forms of NeuroAI and human-like agents.
• Introduces three modes of interaction between the dimensions: generative (mutual support), corroborative (cross-validation), and corrective (error detection).
• Illustrates the rubric with concrete examples from cognitive modeling, neuroscientific hypothesis testing, and machine learning applications.
• Emphasizes iterative feedback loops and rigorous validity testing to ensure credibility across domains.
• Stresses that progress in one dimension does not guarantee progress in others, necessitating explicit alignment and validation.

Experimental results

Research questions

• RQ1 How can researchers clearly distinguish between goals focused on human-like behavior, neural plausibility, and engineering benchmarks in NeuroAI?
• RQ2 What are the differences between weak and strong forms of NeuroAI and human-like agents, and how do they relate to one another?
• RQ3 In what ways can the three dimensions—behavior, neural plausibility, and engineering—interact to generate, corroborate, or correct research findings?
• RQ4 How can interdisciplinary collaboration be structured to ensure validity and mutual advancement across cognitive science, neuroscience, and AI?
• RQ5 What mechanisms are needed to ensure that progress in one domain does not falsely imply progress in another?

Key findings

• The rubric successfully distinguishes between three distinct research commitments: human-like behavior, neural plausibility, and engineering benchmarks.
• Weak NeuroAI focuses on mimicking behavior without neural grounding, while strong NeuroAI integrates neural plausibility with behavioral performance.
• Human-like agents can be weak (behaviorally similar but not neurally plausible) or strong (both behaviorally and neurally aligned with human cognition).
• The three dimensions interact through generative, corroborative, and corrective feedback loops, enabling cross-domain validation and innovation.
• Progress in one domain does not automatically translate to progress in another, necessitating deliberate, iterative validation across domains.
• Future advances in AI will depend on sustained, meticulous, and interdisciplinary feedback loops spanning decades, with outcomes likely to include both known and unforeseen breakthroughs.