[Paper Review] Overlapping neural representations for the position of visible and imagined objects

Amanda K. Robinson, Tijl Grootswagers+4 more|arXiv (Cornell University)|Oct 19, 2020

Visual perception and processing mechanisms4 citations

TL;DR

This study uses EEG and time-resolved multivariate pattern analysis to show that neural representations of imagined object positions overlap with those of visible objects, relying on mid- and high-level visual processing rather than early retinotopic pathways. Imagined positions are decoded earlier and with weaker neural signals than visible ones, indicating top-down, predictive mechanisms underlie internal spatial representations.

ABSTRACT

Humans can covertly track the position of an object, even if the object is temporarily occluded. What are the neural mechanisms underlying our capacity to track moving objects when there is no physical stimulus for the brain to track? One possibility is that the brain 'fills-in' information about imagined objects using internally generated representations similar to those generated by feed-forward perceptual mechanisms. Alternatively, the brain might deploy a higher order mechanism, for example using an object tracking model that integrates visual signals and motion dynamics. In the present study, we used EEG and time-resolved multivariate pattern analyses to investigate the spatial processing of visible and imagined objects. Participants tracked an object that moved in discrete steps around fixation, occupying six consecutive locations. They were asked to imagine that the object continued on the same trajectory after it disappeared and move their attention to the corresponding positions. Time-resolved decoding of EEG data revealed that the location of the visible stimuli could be decoded shortly after image onset, consistent with early retinotopic visual processes. For processing of unseen/imagined positions, the patterns of neural activity resembled stimulus-driven mid-level visual processes, but were detected earlier than perceptual mechanisms, implicating an anticipatory and more variable tracking mechanism. Encoding models revealed that spatial representations were much weaker for imagined than visible stimuli. Monitoring the position of imagined objects thus utilises similar perceptual and attentional processes as monitoring objects that are actually present, but with different temporal dynamics. These results indicate that internally generated representations rely on top-down processes, and their timing is influenced by the predictability of the stimulus.

Motivation & Objective

• To investigate whether neural representations of imagined object positions share mechanisms with those for visible objects.
• To determine the temporal dynamics of neural processing for imagined versus visible stimuli using time-resolved decoding.
• To assess whether internal representations of spatial position rely on perceptual or higher-order attentional mechanisms.
• To control for confounding factors such as eye movements in decoding spatial representations during imagery tasks.

Proposed method

• Participants tracked a moving object across six positions, then imagined it continuing on the same trajectory after disappearance.
• EEG data were collected during both visible tracking and imagined continuation phases.
• Time-resolved multivariate pattern analysis (MVPA) was used to decode spatial position from neural activity patterns.
• A separate pattern estimator with randomized visible stimulus sequences was used to train decoders, ensuring generalization to the tracking task.
• Cross-decoding was performed from the pattern estimator to the tracking task to isolate position-specific neural signals independent of eye movements.
• Posterior and frontal electrode subsets were analyzed separately to validate that spatial decoding originated in visual processing regions.

Experimental results

Research questions

• RQ1 Do neural representations of imagined object positions share spatial coding mechanisms with those for visible objects?
• RQ2 How do the temporal dynamics of neural processing for imagined positions compare to those for visible stimuli?
• RQ3 To what extent do early retinotopic visual processes contribute to imagined object position representation?
• RQ4 Can eye movements account for the observed decoding of spatial positions during imagination tasks?
• RQ5 How does stimulus predictability influence the strength and timing of internal spatial representations?

Key findings

• Neural patterns for imagined object positions were decoded earlier than for visible stimuli, indicating anticipatory processing in internal representations.
• The strength of neural representations for imagined stimuli was significantly weaker than for visible stimuli, as shown by encoding model performance.
• Decoding of spatial position from imagined stimuli relied on mid- and high-level visual processes rather than early retinotopic mechanisms.
• Cross-decoding from a pattern estimator using randomized visible stimuli to the imagined tracking task was above chance, indicating shared neural codes independent of eye movements.
• Posterior electrode analysis confirmed that spatial decoding patterns were consistent with visual system activity, not frontal artifacts or eye movement signals.
• The results suggest that top-down, predictive mechanisms underlie internal spatial representations, with timing modulated by stimulus predictability.