[Paper Review] Instruction-tuning Aligns LLMs to the Human Brain

Khai Loong Aw, Syrielle Montariol+3 more|arXiv (Cornell University)|Dec 1, 2023

Topic Modeling5 citations

TL;DR

This paper investigates whether instruction-tuning enhances alignment between large language models (LLMs) and the human brain. Using brain activity data from fMRI studies and behavioral data from reading times, it finds that instruction-tuned LLMs show a 6.2% average improvement in brain alignment—measured by how well their internal representations predict human neural responses—primarily driven by increased world knowledge and model size, though behavioral alignment remains largely unchanged.

ABSTRACT

Instruction-tuning is a widely adopted finetuning method that enables large language models (LLMs) to generate output that more closely resembles human responses. However, no studies have shown that instruction-tuning actually teaches LLMs to process language in a similar manner as humans. We investigate the effect of instruction-tuning on aligning LLM and human language processing mechanisms in two ways: (1) brain alignment, the similarity of LLM internal representations to neural activity in the human language system, and (2) behavioral alignment, the similarity of LLM and human behavior on a reading task. We assess 25 vanilla and instruction-tuned LLMs on three datasets involving humans reading naturalistic stories and sentences, and find that instruction-tuning generally enhances brain alignment (~6%), but has no similar effect on behavioral alignment. To identify factors underlying this improvement in brain alignment, we compute correlations between brain alignment and various LLM properties, such as model size, problem-solving, and world knowledge understanding. Notably, we find a strong positive correlation between brain alignment and model size (r = 0.95), as well as performance on tasks requiring world knowledge (r = 0.81). Our results demonstrate that instruction-tuning LLMs improves both world knowledge representations and brain alignment, suggesting that the mechanisms that encode world knowledge in LLMs also improve representational alignment to the human brain.

Motivation & Objective

• To determine whether instruction-tuning makes LLMs more similar to human language processing in both neural representations and behavior.
• To assess brain alignment between LLMs and human fMRI responses during naturalistic language comprehension.
• To evaluate behavioral alignment using per-word reading times from human participants.
• To identify model properties that most strongly predict LLM-brain alignment.
• To investigate whether instruction-tuning improves LLMs' representational similarity to the human brain beyond just output mimicry.

Proposed method

• Evaluated 25 LLMs—8 vanilla and 17 instruction-tuned—across three fMRI datasets: Pereira et al. (2018), Blank et al. (2014), and Wehbe et al. (2014).
• Measured brain alignment using Brain-Score, a linear predictivity metric that assesses how well LLM representations predict human fMRI activity.
• Assessed behavioral alignment via Pearson correlation between LLM per-word perplexity and human per-word reading times from the Futrell et al. (2018) dataset.
• Computed model properties including next-word prediction (NWP) performance, model size, and performance on BBH (Big-Bench Hard) and MMLU (Massive Multi-task Language Understanding) benchmarks.
• Used Pearson correlation to analyze relationships between LLM properties and brain alignment.
• Replicated results using open-source models and public repositories (Brain-Score and instruct-eval) for full reproducibility.

Experimental results

Research questions

• RQ1 Does instruction-tuning improve the similarity between LLM internal representations and human brain activity during language processing?
• RQ2 Does instruction-tuning enhance behavioral similarity between LLMs and humans in reading comprehension tasks?
• RQ3 Which model properties—such as size, world knowledge, or reasoning ability—are most predictive of LLM-brain alignment?
• RQ4 Is the improvement in brain alignment due to enhanced world knowledge or other capabilities?
• RQ5 How does the performance of LLMs on reasoning and knowledge benchmarks correlate with their alignment to human neural responses?

Key findings

• Instruction-tuning increases brain alignment by an average of 6.2% across tested LLMs and neural datasets.
• There is a strong positive correlation between model size and brain alignment (r = 0.95).
• World knowledge, as measured by MMLU performance, shows a high positive correlation with brain alignment (r = 0.81).
• Instruction-tuning does not significantly improve behavioral alignment with human reading times, despite its effect on brain alignment.
• Next-word prediction ability does not show a consistent correlation with brain alignment in modern Transformer-based LLMs.
• The results suggest that world knowledge and model scale are key drivers of representational alignment to the human brain.