[Paper Review] Persona-Driven Benchmarking for Generalizable and Human-Aware Artificial General Intelligence

Shunyu Yao, Jeffrey Zhao+5 more|arXiv (Cornell University)|Oct 6, 2022

Topic Modeling518 citations

TL;DR

This paper introduces  model, a prompt-based paradigm that interleaves reasoning traces and actions in large language models to improve performance and interpretability across QA, fact verification, and interactive decision-making tasks, outperforming baselines and enabling better human alignment.

ABSTRACT

This research paper, "Persona-Driven Benchmarking for Generalizable and Human-Aware Artificial General Intelligence," proposes a novel architectural solution to transition current Large Language Models (LLMs) from sophisticated pattern-matchers to genuine Artificial General Intelligence (AGI) agents. The core argument of the paper is that achieving AGI is primarily an architectural challenge (creating a "gearbox" of metacognitive functions) rather than solely a scaling or fundamental modeling problem (the "engine"). The proposed framework is an architectural overlay composed of three synergistic components that address the key limitations of contemporary LLMs: lack of persistent memory, poor accountability, and narrow human-centric reasoning. Key Components of the Proposed Architecture Persona Module (for Dynamic Contextualization and Self-Awareness): Function: Handles the dynamic contextualization of all interactions, performs causal intent inference regarding the user's needs, and manages the AGI's evolving, internal self-identity (or "self-model"). Goal: To enable the AGI to move beyond simple response generation to goal-oriented optimization based on a deep, human-aware understanding of the user and its own capabilities. Benchmarking Loop (for Metacognitive Self-Correction): Function: This component acts as the AGI's metacognitive self-correction engine. It rigorously audits the LLM's output against a goal-specific metric, performs causal inference to determine why a failure occurred, and then dictates a permanent learning path. Goal: To foster an internal, continuous learning cycle that ensures the AGI is accountable and capable of generalized skill transfer from one task to a similar, new one. FedRAG (Federated Reflective Augmented Generation): Function: A novel, persistent external memory layer that functions as a dynamic knowledge graph. It is crucial because it decouples learned skills and state from the static LLM weights. It stores successful outcomes, critical causal inferences from the Benchmarking Loop, user preferences, and Persona state updates. Goal: To provide the AGI with persistent, generalizable memory and ensure that learning is permanent, efficient, and does not lead to catastrophic forgetting or excessive model scaling. Central Thesis The paper asserts that the integrated, recursive loop formed by these three components—where the Benchmarking Loop's reflective output is written to the FedRAG memory for use by the Persona Module in future interactions—is the necessary architectural scaffold to achieve true, generalizable, and human-aware AGI.

Motivation & Objective

• Motivate tight coupling of reasoning and acting to improve robustness and interpretability in LLM-driven tasks.
• Show that interleaved reasoning traces and actions can outperform reasoning-only or action-only baselines on diverse benchmarks.
• Demonstrate improved interpretability, trustworthiness, and diagnosability through visible reasoning trajectories.
• Assess how external knowledge retrieval and internal reasoning complement each other in knowledge-intensive tasks.

Proposed method

• Augment the action space of an agent to include a language-based reasoning trace that updates the context for future steps.
• Prompt a frozen PaLM-540B (and GPT-3 in Appendix) with few-shot trajectories that include thoughts, actions, and observations.
• Use a simple external knowledge API (Wikipedia) with actions like search, lookup, and finish to ground reasoning in facts.
• Compare  model against Standard, CoT, CoT-SC, and Act baselines across multiple tasks.
• Propose hybrid prompting by switching between  model and CoT-SC based on task feedback to leverage internal and external knowledge.
• Explore finetuning with 3,000  model trajectories on HotpotQA to improve decoding of thoughts, actions, and observations.

Experimental results

Research questions

• RQ1 Can interleaving reasoning traces with actions improve performance and interpretability over reasoning-only or action-only baselines?
• RQ2 Does interacting with external knowledge bases (e.g., Wikipedia) improve grounded reasoning and reduce hallucinations?
• RQ3 What is the impact of combining internal reasoning with external knowledge on QA and fact verification tasks?
• RQ4 How does  model perform on interactive decision-making benchmarks compared to imitation and reinforcement learning baselines?
• RQ5 Does finetuning enhance the model's ability to decode trajectories and generalize to new tasks?

Key findings

•  model outperforms action-only baselines on HotpotQA and Fever, and combining  model with CoT-SC yields best prompting results in some settings.
•  model demonstrates reduced hallucination and improved grounding by accessing a simple Wikipedia API for relevant facts.
• On ALFWorld and WebShop,  model achieves significant absolute improvements (34% and 10%, respectively) over stronger baselines with limited in-context examples.
•  model offers interpretable trajectories (thoughts, actions, observations) that humans can inspect to diagnose and trust decisions.
• Prompting alone scales modestly, but finetuning with a few thousand trajectories makes  model the best method among evaluated options on HotpotQA.
•  model generally provides a robust and generalizable framework across diverse domains, with potential for further gains when combined with reinforcement learning and multi-task training.