[Paper Review] Talking Space: inference from spatial linguistic meanings

Vincent Wang-Maścianica, Bob Coecke|arXiv (Cornell University)|Sep 14, 2021

Language, Metaphor, and Cognition39 references4 citations

TL;DR

This paper proposes a compositional, diagrammatic framework that integrates spatial linguistic meanings with physical space using DisCoCat and DisCoCirc, enabling inference over spatial relations (e.g., 'above', 'next to') and agent-specific spatial capabilities (e.g., chesspiece moves). The key contribution is a mathematically rigorous, implementation-agnostic model that supports spatial reasoning in natural language, unifying linguistic structure with spatial semantics and embodiment.

ABSTRACT

This paper concerns the intersection of natural language and the physical space around us in which we live, that we observe and/or imagine things within. Many important features of language have spatial connotations, for example, many prepositions (like in, next to, after, on, etc.) are fundamentally spatial. Space is also a key factor of the meanings of many words/phrases/sentences/text, and space is a, if not the key, context for referencing (e.g. pointing) and embodiment. We propose a mechanism for how space and linguistic structure can be made to interact in a matching compositional fashion. Examples include Cartesian space, subway stations, chesspieces on a chess-board, and Penrose's staircase. The starting point for our construction is the DisCoCat model of compositional natural language meaning, which we relax to accommodate physical space. We address the issue of having multiple agents/objects in a space, including the case that each agent has different capabilities with respect to that space, e.g., the specific moves each chesspiece can make, or the different velocities one may be able to reach. Once our model is in place, we show how inferences drawing from the structure of physical space can be made. We also how how linguistic model of space can interact with other such models related to our senses and/or embodiment, such as the conceptual spaces of colour, taste and smell, resulting in a rich compositional model of meaning that is close to human experience and embodiment in the world.

Motivation & Objective

• To formalize the interaction between linguistic structure and physical space in a compositional, diagrammatic framework.
• To enable inference over spatial relations (e.g., 'above', 'next to') using compositional semantics.
• To model agent-specific spatial capabilities (e.g., chesspiece movement) within a unified semantic framework.
• To integrate spatial meaning with other conceptual spaces (e.g., color, taste) for richer, embodied meaning representation.
• To bridge linguistic semantics with physical space in a way that supports systematic, productive concept formation.

Proposed method

• Extends the DisCoCat model by relaxing it to incorporate physical space, using pregroups for syntactic structure and diagrammatic reasoning.
• Introduces spatial relations as compositional entities that can be combined across sentences via DisCoCirc, enabling multi-agent spatial inference.
• Models spatial features such as object extent and agent-specific movement capabilities (e.g., knight moves in chess) as internal wirings in the semantic category.
• Uses Cartesian space R³ as a universal embedding for diverse spatial domains (e.g., grids, subway lines), enabling unification across discrete and continuous spaces.
• Employs a compositional, category-theoretic approach that preserves grammatical structure and allows for learning spatial relations via statistical or handcrafted methods.
• Supports interaction between linguistic meaning and sensory conceptual spaces (e.g., color, smell) through compositional compositionality.

Experimental results

Research questions

• RQ1 How can spatial linguistic meanings be composed in a way that respects the physical structure of space and enables inference?
• RQ2 How can agent-specific spatial capabilities (e.g., chesspiece movement) be encoded and composed within a linguistic semantic framework?
• RQ3 How can spatial reasoning in language be formalized using compositional category-theoretic methods that match linguistic syntax?
• RQ4 What is the relationship between DisCoCat models and Montague semantics when spatial and embodied meaning are included?
• RQ5 How can spatial meaning be integrated with other conceptual spaces (e.g., color, taste) to form a richer, embodied model of meaning?

Key findings

• The model successfully enables inference such as transitivity of spatial prepositions (e.g., 'above' in 'painting above chest' implies 'light above chest').
• It supports reasoning about multiple agents in space, including cases where agents have distinct spatial capabilities (e.g., a knight's L-shaped moves in chess).
• The framework allows for the specification of a specific piece on a chessboard via a linguistic diagram, demonstrating explicit spatial reasoning capability.
• The model is compatible with both handcrafted and learned spatial relations, supporting flexibility and scalability.
• It provides a compositional bridge between linguistic structure and physical space, enabling systematic, productive concept formation.
• The approach unifies spatial semantics with other conceptual spaces (e.g., color, taste) through a common compositional architecture, enhancing embodiment in meaning representation.