[Paper Review] Bridging the Human-AI Knowledge Gap: Concept Discovery and Transfer in AlphaZero

Lisa Schut, Nenad Tomašev+4 more|arXiv (Cornell University)|Oct 25, 2023

Sports Analytics and Performance8 citations

TL;DR

The paper introduces a framework to extract machine-unique (M-H) chess concepts from AlphaZero, and demonstrates that top grandmasters can learn and apply these concepts through concept prototypes and teachability assessments.

ABSTRACT

Artificial Intelligence (AI) systems have made remarkable progress, attaining super-human performance across various domains. This presents us with an opportunity to further human knowledge and improve human expert performance by leveraging the hidden knowledge encoded within these highly performant AI systems. Yet, this knowledge is often hard to extract, and may be hard to understand or learn from. Here, we show that this is possible by proposing a new method that allows us to extract new chess concepts in AlphaZero, an AI system that mastered the game of chess via self-play without human supervision. Our analysis indicates that AlphaZero may encode knowledge that extends beyond the existing human knowledge, but knowledge that is ultimately not beyond human grasp, and can be successfully learned from. In a human study, we show that these concepts are learnable by top human experts, as four top chess grandmasters show improvements in solving the presented concept prototype positions. This marks an important first milestone in advancing the frontier of human knowledge by leveraging AI; a development that could bear profound implications and help us shape how we interact with AI systems across many AI applications.

Motivation & Objective

• Motivate and formalize the goal of learning from super-human AI knowledge to extend human expertise.
• Define and operationalize concepts as units of knowledge that can be transferred and taught.
• Develop a framework to discover, filter, and validate M-H concepts in AlphaZero.
• Demonstrate teachability and novelty of discovered concepts through human and AI experiments.

Proposed method

• Formulate concept discovery as a convex optimization problem to extract sparse concept vectors in latent spaces.
• Differentiate static concepts (single-state) and dynamic concepts (state sequences) and derive corresponding constraints.
• Use AlphaZero’s policy-value network and MCTS rollouts to contrast positive (AZ-preferred) and negative (suboptimal) trajectories for dynamic concepts.
• Filter concepts for teachability by measuring transfer to a student network using concept prototypes and KL-divergence of policies.
• Assess novelty by ensuring concepts do not arise in human games and by leveraging late-stage AZ training differences (Elo-based) to identify complex concepts.
• Generate concept prototypes from Xproto and train a student to imitate teacher moves, evaluating concept transfer via top-1 move agreement.

Experimental results

Research questions

• RQ1 Do AlphaZero's latent representations contain machine-unique knowledge (M-H) that can be meaningfully learned by humans?
• RQ2 Can we extract, filter, and teach concepts from AlphaZero that are novel relative to human chess knowledge?
• RQ3 Are concept prototypes effective in transferring AI-derived concepts to humans or other AI agents?
• RQ4 How do dynamic, MCTS-driven concepts differ from static, single-position concepts in terms of learnability and transfer?

Key findings

• Grandmasters showed improved ability to find concept-based moves aligned with AlphaZero after exposure to concept prototypes.
• Concepts discovered often combine chess ideas in ways that deviate from traditional human principles, suggesting different position-concept relationships.
• Teachability experiments indicate concept prototypes enable faster and more reliable transfer to a student network than random AZ/game positions.
• The framework can reveal and relate new concepts to human-labeled concepts via graph analysis, providing interpretability of the learned notions.
• Concept novelty is ensured by focusing on late-stage AZ concepts and by comparing AZ vs. human game representations to identify machine-unique knowledge.