[Paper Review] Structural Similarities Between Language Models and Neural Response Measurements

Jiaang Li, Antonia Karamolegkou+4 more|arXiv (Cornell University)|Jun 2, 2023

Topic Modeling42 references4 citations

TL;DR

This paper demonstrates that as large language models (LLMs) scale in size, their internal word representations become increasingly structurally similar to neural response patterns measured via fMRI during language comprehension. Using ridge regression, representational similarity analysis (RSA), and Procrustes analysis, the authors show that LLMs achieve up to 25% precision-at-10 in decoding words from fMRI data—significantly above random chance—indicating a strong geometric alignment between artificial and human neural representations of language.

ABSTRACT

Large language models (LLMs) have complicated internal dynamics, but induce representations of words and phrases whose geometry we can study. Human language processing is also opaque, but neural response measurements can provide (noisy) recordings of activation during listening or reading, from which we can extract similar representations of words and phrases. Here we study the extent to which the geometries induced by these representations, share similarities in the context of brain decoding. We find that the larger neural language models get, the more their representations are structurally similar to neural response measurements from brain imaging. Code is available at \url{https://github.com/coastalcph/brainlm}.

Motivation & Objective

• To investigate whether the representational geometry of large language models (LLMs) aligns with human neural responses during language processing.
• To determine if increasing model size enhances structural similarity between LLM representations and fMRI-measured brain activity.
• To evaluate the feasibility of decoding linguistic concepts from fMRI signals using LLM-based representations.
• To explore the philosophical and cognitive implications of this alignment, particularly regarding semantic grounding.
• To assess the role of model depth in achieving better alignment with human neural representations.

Proposed method

• The study compares word representations from three LLM families (including BERT) and one static embedding method against fMRI data from two publicly available datasets: the Harry Potter and Natural Stories datasets.
• Representational similarity analysis (RSA) is used to quantify geometric similarity between LLM and fMRI representation spaces.
• Ridge regression is applied to predict fMRI responses from LLM representations, enabling evaluation of decoding performance via precision-at-k (P@k).
• Procrustes analysis is used to assess isomorphism between LLM and fMRI representations when dimensionalities match (d = d’).
• Gaussian smoothing is applied to fMRI data to mitigate noise and improve signal-to-noise ratio.
• The analysis is performed across multiple model sizes and layers to assess depth-dependent alignment trends.

Figure 1: Experimental flow and main results. We run experiments with three families of LLMs (comparing LLMs of different sizes within families), two fMRI datasets, and three projection algorithms, and results are the same across all combinations: LLMs converge toward human-like representations, ena

Experimental results

Research questions

• RQ1 To what extent do the representational geometries of large language models resemble those of human neural responses during language comprehension?
• RQ2 How does increasing model size affect the structural similarity between LLM representations and fMRI-measured brain activity?
• RQ3 Does the alignment between LLM and neural representations improve with model depth, particularly in later layers?
• RQ4 Can word-level decoding from fMRI signals be achieved using LLM representations, and how does performance compare to random baseline?
• RQ5 What is the philosophical significance of the observed isomorphism between artificial and biological language representations?

Key findings

• The largest LLMs achieve a precision-at-10 (P@10) of up to 25% in decoding words from fMRI signals, far exceeding the random baseline of less than 1%.
• Structural similarity between LLM representations and fMRI responses increases consistently with model size across all three LLM families studied.
• Deeper layers in LLMs show stronger alignment with human neural representations, and this improvement does not plateau, suggesting continued gains with increased depth.
• The alignment is robust across two distinct fMRI datasets and three evaluation methods: ridge regression, RSA, and Procrustes analysis.
• The observed isomorphism is not trivial, as it specifically aligns the nearest-neighbor graph structures of LLM and fMRI representations, supporting the idea of semantic grounding.
• The results suggest that semantic and pragmatic representations in LLMs are particularly aligned with human brain responses, while syntactic representations in earlier layers show weaker correspondence.

Figure 2: Convergence Results for Three Families of Language Models on Two Datasets . The task for Harry Potter dataset here is: Given a neural response, which word in a vocabulary of 1,291 words, was read at the time the response was recorded? Random retrieval baseline P@10 is less than $1\%$ , whi