[Paper Review] Demystifying Neural Language Models' Insensitivity to Word-Order

Louis Clouâtre, Prasanna Parthasarathi+2 more|arXiv (Cornell University)|Jul 29, 2021

Topic Modeling67 references4 citations

TL;DR

This paper introduces two novel metrics—Direct Neighbour Displacement (DND) and Index Displacement Count (IDC)—to quantify local and global word-order perturbations in text, revealing that neural language models rely more on local token ordering than global reordering. The study demonstrates that even state-of-the-art models like Transformers, LSTMs, and CNNs are significantly more sensitive to local ordering disruptions than to global reordering, challenging assumptions about their syntactic sensitivity.

ABSTRACT

Recent research analyzing the sensitivity of natural language understanding models to word-order perturbations have shown that the state-of-the-art models in several language tasks may have a unique way to understand the text that could seldom be explained with conventional syntax and semantics. In this paper, we investigate the insensitivity of natural language models to word-order by quantifying perturbations and analysing their effect on neural models' performance on language understanding tasks in GLUE benchmark. Towards that end, we propose two metrics - the Direct Neighbour Displacement (DND) and the Index Displacement Count (IDC) - that score the local and global ordering of tokens in the perturbed texts and observe that perturbation functions found in prior literature affect only the global ordering while the local ordering remains relatively unperturbed. We propose perturbations at the granularity of sub-words and characters to study the correlation between DND, IDC and the performance of neural language models on natural language tasks. We find that neural language models - pretrained and non-pretrained Transformers, LSTMs, and Convolutional architectures - require local ordering more so than the global ordering of tokens. The proposed metrics and the suite of perturbations allow a systematic way to study the (in)sensitivity of neural language understanding models to varying degree of perturbations.

Motivation & Objective

• To investigate why state-of-the-art neural language models exhibit insensitivity to word-order perturbations in natural language understanding tasks.
• To quantify the impact of local versus global word-order changes on model performance using new, fine-grained perturbation techniques.
• To develop metrics that distinguish between local (neighboring token) and global (overall sequence) ordering disruptions in text.
• To evaluate the sensitivity of diverse architectures—Transformers, LSTMs, CNNs—across pre-trained and non-pretrained variants to these perturbations.
• To provide a systematic framework for analyzing model robustness to varying degrees of word-order manipulation.

Proposed method

• Propose the Direct Neighbour Displacement (DND) metric to measure disruptions in adjacent token pairs, capturing local ordering changes.
• Introduce the Index Displacement Count (IDC) metric to quantify global reordering by measuring how far each token deviates from its original position.
• Design sub-word and character-level perturbations to enable fine-grained control over local and global ordering changes.
• Apply these perturbations to GLUE benchmark datasets to evaluate performance degradation across multiple NLP tasks.
• Train and evaluate a range of models—including pretrained and non-pretrained Transformers, LSTMs, and CNNs—under controlled perturbation conditions.
• Correlate DND and IDC scores with model performance drops to assess relative sensitivity to local vs. global ordering.

Experimental results

Research questions

• RQ1 How do local and global word-order perturbations differentially affect the performance of neural language models?
• RQ2 To what extent do state-of-the-art models rely on local token ordering compared to global sequence structure?
• RQ3 Can sub-word and character-level perturbations reveal hidden sensitivities in model behavior not exposed by sentence-level reordering?
• RQ4 How do DND and IDC metrics correlate with performance degradation across diverse architectures and tasks?
• RQ5 Do pretrained models exhibit greater insensitivity to word-order changes than non-pretrained models?

Key findings

• Neural language models, including Transformers, LSTMs, and CNNs, are significantly more sensitive to local ordering changes than to global reordering, as measured by DND and IDC.
• Perturbations in prior work primarily affect global ordering while preserving local token adjacency, which may explain the observed insensitivity in model performance.
• Sub-word and character-level perturbations reveal that local ordering is a critical signal for model understanding, even in large pre-trained models.
• The proposed DND and IDC metrics effectively capture the distinction between local and global ordering disruptions and correlate strongly with performance drops.
• Performance degradation is consistently higher under DND-increasing perturbations than under IDC-increasing ones, indicating stronger reliance on local structure.
• The findings suggest that current models may rely more on local sequential patterns than on syntactic or semantic global structure, challenging assumptions about their syntactic understanding.