[Paper Review] Experimental Support for a Categorical Compositional Distributional Model of Meaning

Edward Grefenstette, Mehrnoosh Sadrzadeh|arXiv (Cornell University)|Jun 20, 2011

Natural Language Processing Techniques24 references223 citations

TL;DR

This paper presents a practical implementation of a categorical compositional distributional model of meaning that combines distributional semantics with type-logical grammar. By learning matrices for relational words (e.g., verbs, adjectives) from the British National Corpus and applying them to word vectors, the model achieves state-of-the-art performance on intransitive verb disambiguation and outperforms existing models on a novel transitive sentence disambiguation task, demonstrating improved syntactic sensitivity and scalability with increasing sentence complexity.

ABSTRACT

Modelling compositional meaning for sentences using empirical distributional methods has been a challenge for computational linguists. We implement the abstract categorical model of Coecke et al. (arXiv:1003.4394v1 [cs.CL]) using data from the BNC and evaluate it. The implementation is based on unsupervised learning of matrices for relational words and applying them to the vectors of their arguments. The evaluation is based on the word disambiguation task developed by Mitchell and Lapata (2008) for intransitive sentences, and on a similar new experiment designed for transitive sentences. Our model matches the results of its competitors in the first experiment, and betters them in the second. The general improvement in results with increase in syntactic complexity showcases the compositional power of our model.

Motivation & Objective

• To bridge the gap between formal compositional semantics and distributional semantics by unifying logical structure with empirical word vectors.
• To develop a scalable, unsupervised method for learning relational word matrices (e.g., verbs) from corpus data.
• To evaluate the model’s compositional power on increasingly complex syntactic structures, particularly transitive sentences.
• To demonstrate that the model’s sensitivity to grammatical structure leads to better alignment with human judgments than commutative composition models.
• To provide experimental validation of a high-level categorical framework for compositional distributional meaning in a real-world corpus setting.

Proposed method

• The model represents words as vectors (nouns) or matrices (relational words like verbs and adjectives), based on co-occurrence statistics from the British National Corpus (BNC).
• It applies relational matrices to their arguments via tensor contraction (Kronecker product and matrix-vector multiplication), following the formalism of pregroup grammars and category theory.
• The model uses a bottom-up learning approach: matrices are trained directly from sentence-level context data, not via regression from composite vectors.
• For evaluation, it applies the model to two disambiguation tasks: one for intransitive verbs (from prior work) and a new one for transitive verbs with controlled syntactic complexity.
• Performance is measured using correlation with human-annotated similarity judgments, comparing the model against baseline and multiplicative models.
• The model’s compositional structure ensures uniform scalability to complex sentences and enables comparison of sentences with different syntactic forms.

Experimental results

Research questions

• RQ1 Can a high-level categorical compositional distributional model be practically implemented using real-world corpus data?
• RQ2 Does the model’s compositional structure lead to improved performance on disambiguation tasks compared to non-compositional or commutative models?
• RQ3 How does the model’s performance scale with increasing syntactic complexity, particularly in transitive constructions?
• RQ4 To what extent does the model’s sensitivity to grammatical structure align with human judgments of semantic similarity?
• RQ5 Can the model effectively represent relational words (e.g., verbs) as matrices that act on their arguments in a way that preserves syntactic and semantic distinctions?

Key findings

• The model matches the performance of leading competitors on the intransitive verb disambiguation task, confirming its viability on simpler syntactic structures.
• On the novel transitive sentence disambiguation task, the model significantly outperforms both the baseline and multiplicative models, demonstrating superior syntactic sensitivity.
• The model shows a consistent increase in alignment with human annotator judgments as sentence complexity increases, indicating strong compositional power.
• The difference in performance between the categorical model and baseline models is smaller in mean correlation, but the categorical model shows better alignment with human judgments, especially in complex cases.
• The results confirm that the model’s compositional architecture enables scalable and robust meaning composition across diverse syntactic constructions.
• The model’s ability to learn relational word matrices from raw corpus data without supervision supports its practical feasibility and generalization potential.