[Paper Review] Language Writ Large: LLMs, ChatGPT, Grounding, Meaning and Understanding

Stevan Harnad|arXiv (Cornell University)|Feb 3, 2024

Artificial Intelligence in Healthcare and Education6 citations

TL;DR

The paper argues that LLMs like ChatGPT do not truly understand; it discusses benign biases at LLM scale, grounding gaps, and proposes hunches about how indirect verbal grounding, circularity, and other factors shape their capabilities, framed as a dialogue with ChatGPT-4.

ABSTRACT

Apart from what (little) OpenAI may be concealing from us, we all know (roughly) how ChatGPT works (its huge text database, its statistics, its vector representations, and their huge number of parameters, its next-word training, and so on). But none of us can say (hand on heart) that we are not surprised by what ChatGPT has proved to be able to do with these resources. This has even driven some of us to conclude that ChatGPT actually understands. It is not true that it understands. But it is also not true that we understand how it can do what it can do. I will suggest some hunches about benign biases: convergent constraints that emerge at LLM scale that may be helping ChatGPT do so much better than we would have expected. These biases are inherent in the nature of language itself, at LLM scale, and they are closely linked to what it is that ChatGPT lacks, which is direct sensorimotor grounding to connect its words to their referents and its propositions to their meanings. These convergent biases are related to (1) the parasitism of indirect verbal grounding on direct sensorimotor grounding, (2) the circularity of verbal definition, (3) the mirroring of language production and comprehension, (4) iconicity in propositions at LLM scale, (5) computational counterparts of human categorical perception in category learning by neural nets, and perhaps also (6) a conjecture by Chomsky about the laws of thought. The exposition will be in the form of a dialogue with ChatGPT-4.

Motivation & Objective

• Assess whether ChatGPT-like LLMs possess genuine understanding or meaning.
• Identify convergent biases that emerge at LLM scale that influence performance.
• Examine how lack of direct sensorimotor grounding affects referents and propositions.
• Propose hypotheses about grounding, definition, and representation in large language models.

Proposed method

• Present a dialogue with ChatGPT-4 to explore the topic.
• Discuss a set of convergent biases that may emerge at LLM scale.
• Outline connections between indirect verbal grounding, circularity of definition, and perceptual categorization.
• Link arguments to broader theories of grounding and language understanding.

Experimental results

Research questions

• RQ1 Do ChatGPT-style LLMs truly understand the meanings of their outputs?
• RQ2 What convergent biases emerge at LLM scale that enhance or limit performance?
• RQ3 How does the absence of direct sensorimotor grounding influence referents, meanings, and propositions in LLMs?
• RQ4 What theoretical perspectives (e.g., Chomsky, categorization, iconicity) illuminate LLM capabilities and limitations?

Key findings

• LLMs can perform impressive tasks without genuine understanding or grounding in sensorimotor experience.
• Convergent biases at scale may arise from the structure and distribution of language data and model architecture.
• Indirect verbal grounding appears to parasitize direct sensorimotor grounding, impacting referential connections.
• Circularity of verbal definitions and mirroring between production and comprehension are relevant to LLM behavior.
• Iconicity and category learning principles may correspond to LLM-scale representations, offering partial explanations for performance.