[Paper Review] Category Theory for Quantum Natural Language Processing

Alexis Toumi|arXiv (Cornell University)|Dec 13, 2022

Computational Physics and Python Applications4 citations

TL;DR

This paper introduces a category-theoretic framework for quantum natural language processing (QNLP) by modeling syntactic structures using free categories over signatures, enabling compositional semantics through path-based arrows. The key contribution is a formal isomorphism between functors from free categories and signature morphisms, establishing a foundation for quantum circuit-like data pipelines in NLP.

ABSTRACT

This thesis introduces quantum natural language processing (QNLP) models based on a simple yet powerful analogy between computational linguistics and quantum mechanics: grammar as entanglement. The grammatical structure of text and sentences connects the meaning of words in the same way that entanglement structure connects the states of quantum systems. Category theory allows to make this language-to-qubit analogy formal: it is a monoidal functor from grammar to vector spaces. We turn this abstract analogy into a concrete algorithm that translates the grammatical structure onto the architecture of parameterised quantum circuits. We then use a hybrid classical-quantum algorithm to train the model so that evaluating the circuits computes the meaning of sentences in data-driven tasks. The implementation of QNLP models motivated the development of DisCoPy (Distributional Compositional Python), the toolkit for applied category theory of which the first chapter gives a comprehensive overview. String diagrams are the core data structure of DisCoPy, they allow to reason about computation at a high level of abstraction. We show how they can encode both grammatical structures and quantum circuits, but also logical formulae, neural networks or arbitrary Python code. Monoidal functors allow to translate these abstract diagrams into concrete computation, interfacing with optimised task-specific libraries. The second chapter uses DisCopy to implement QNLP models as parameterised functors from grammar to quantum circuits. It gives a first proof-of-concept for the more general concept of functorial learning: generalising machine learning from functions to functors by learning from diagram-like data. In order to learn optimal functor parameters via gradient descent, we introduce the notion of diagrammatic differentiation: a graphical calculus for computing the gradients of parameterised diagrams.

Motivation & Objective

• To formalize quantum natural language processing using category theory, particularly free categories over signatures.
• To model syntactic structures in NLP as paths in directed multigraphs (signatures), capturing data pipelines compositionally.
• To establish a natural isomorphism between functors from free categories and morphisms of signatures, enabling compositional semantics.
• To provide a categorical foundation for quantum circuits and NLP that mirrors the role of lists as free monoids in programming.
• To implement this framework in DisCoPy, a Python library for quantum algorithms and NLP, using category-theoretic abstractions.

Proposed method

• Define a signature as a tuple (Σ₀, Σ₁, dom, cod), where Σ₀ is a set of generating objects and Σ₁ a set of generating arrows (boxes) with domain and codomain maps.
• Construct the free category F(Σ) over a signature Σ, where objects are elements of Σ₀ and arrows are finite paths of composable generating arrows.
• Define a functor Sig → Cat that maps signatures to their free categories, preserving morphisms via relabeling of generating arrows.
• Establish an adjunction between the forgetful functor Cat → Sig and the free category functor Sig → Cat, with unit sending a box to its trivial path and counit composing paths.
• Use the natural isomorphism Cat(F(Σ), C) ≃ Sig(Σ, U(C)) to show that defining a functor from a free category is equivalent to defining a signature morphism.
• Implement the framework in DisCoPy using Python classes for categories and signatures, leveraging decorators and immutable lists to model paths and morphisms.

Experimental results

Research questions

• RQ1 How can category theory be used to model compositional semantics in quantum natural language processing?
• RQ2 What is the categorical structure underlying data pipelines in NLP, and how does it relate to quantum circuits?
• RQ3 Can the construction of free categories over signatures serve as a formal foundation for compositional NLP, analogous to free monoids for lists?
• RQ4 What is the role of adjunctions in connecting signature morphisms and functors from free categories?
• RQ5 How can this framework be implemented efficiently in a programming language like Python for practical quantum NLP applications?

Key findings

• The free category construction over a signature provides a categorical model for data pipelines in NLP, where arrows represent sequences of operations.
• There exists a natural isomorphism between functors from a free category F(Σ) to a category C and morphisms of signatures from Σ to the underlying signature of C.
• The unit of the adjunction sends each generating arrow (box) in a signature to the trivial path of length one, while the counit composes paths into a single arrow.
• The adjunction between Sig and Cat establishes that free categories are the left adjoint to the forgetful functor from categories to signatures, generalizing the free monoid construction.
• The framework is implemented in DisCoPy, demonstrating that path-based arrows in free categories can be natively represented and manipulated in Python using immutable lists and decorators.
• The isomorphism Cat(F(Σ), C) ≃ Sig(Σ, U(C)) confirms that defining a functor from a free category is equivalent to specifying a signature morphism, enabling modular and compositional semantics.