[Paper Review] Perceptual similarity of visual patterns predicts the similarity of their dynamic neural activation patterns measured with MEG

Susan G. Wardle, Nikolaus Kriegeskorte+3 more|arXiv (Cornell University)|Jun 7, 2015

Visual perception and processing mechanisms52 references4 citations

TL;DR

This study demonstrates that perceptual similarity of abstract visual patterns strongly predicts the similarity of their dynamic neural activation patterns in the human brain, as measured by magnetoencephalography (MEG). Using Gabor patch-based stimuli with unique global forms, the authors show that early neural responses are driven by retinotopic organization, but by 80ms post-stimulus, perceptual similarity becomes the strongest predictor of neural representational similarity—reaching the noise ceiling limit of correlation with neural data.

ABSTRACT

Perceptual similarity is a cognitive judgment that represents the end-stage of a complex cascade of hierarchical processing throughout visual cortex. Previous studies have shown a correspondence between the similarity of coarse-scale fMRI activation patterns and the perceived similarity of visual stimuli, suggesting that visual objects that appear similar also share similar underlying patterns of neural activation. Here we explore the temporal relationship between the human brain's time-varying representation of visual patterns and behavioral judgments of perceptual similarity. The visual stimuli were abstract patterns constructed from identical perceptual units (oriented Gabor patches) so that each pattern had a unique global form or perceptual 'Gestalt'. The visual stimuli were decodable from evoked neural activation patterns measured with magnetoencephalography (MEG), however, stimuli differed in the similarity of their neural representation as estimated by differences in decodability. Early after stimulus onset (from 50ms), a model based on retinotopic organization predicted the representational similarity of the visual stimuli. Following the peak correlation between the retinotopic model and neural data at 80ms, the neural representations quickly evolved so that retinotopy no longer provided a sufficient account of the brain's time-varying representation of the stimuli. Overall the strongest predictor of the brain's representation was a model based on human judgments of perceptual similarity, which reached the limits of the maximum correlation with the neural data defined by the 'noise ceiling'. Our results show that large-scale brain activation patterns contain a neural signature for the perceptual Gestalt of composite visual features, and demonstrate a strong correspondence between perception and complex patterns of brain activity.

Motivation & Objective

• To investigate how dynamic neural representations in the human brain relate to perceptual similarity of visual stimuli.
• To determine whether perceptual similarity judgments predict the similarity of time-varying neural activation patterns measured with MEG.
• To compare the predictive power of retinotopic models versus perceptual similarity models for neural representations over time.
• To assess the extent to which neural representational similarity is constrained by the noise ceiling of measurement reliability.

Proposed method

• Stimuli were abstract visual patterns constructed from identical oriented Gabor patches to ensure consistent perceptual units while varying global Gestalt form.
• MEG was used to record time-resolved neural activation patterns in response to each stimulus, enabling high-temporal-resolution analysis of neural dynamics.
• Representational similarity analysis (RSA) was applied to compare neural activation patterns across stimuli, quantifying similarity in neural responses.
• A retinotopic model was constructed based on spatial retinotopic organization to predict early neural responses.
• A perceptual similarity model was derived from human behavioral judgments of stimulus similarity to predict neural representational similarity.
• Correlation between model predictions and neural data was computed at multiple time points, with noise ceiling analysis used to establish the maximum achievable correlation.

Experimental results

Research questions

• RQ1 How does the similarity of neural activation patterns in the human brain relate to the perceptual similarity of visual stimuli over time?
• RQ2 To what extent does retinotopic organization predict neural representations of visual patterns in early processing stages?
• RQ3 Does a model based on human perceptual similarity judgments better predict neural representational similarity than retinotopic models?
• RQ4 At what time point does perceptual similarity become the dominant predictor of neural representation?

Key findings

• Neural representational similarity of visual patterns, as measured by MEG, closely tracks perceptual similarity judgments made by human observers.
• From 50ms after stimulus onset, a retinotopic model predicted neural responses, peaking in correlation at 80ms post-stimulus.
• After 80ms, the retinotopic model's predictive power declined, indicating that higher-level processing supersedes early retinotopic organization.
• The perceptual similarity model achieved the highest correlation with neural data, reaching the noise ceiling limit, indicating it explains nearly all reliable variance in neural representations.
• The results demonstrate that large-scale brain activation patterns encode the perceptual Gestalt of composite visual features.
• The study provides strong evidence for a direct link between perceptual experience and complex, dynamic patterns of neural activity in human visual cortex.